How to Run the Karpathy Loop: AI-Automated Benchmarking That Made Shopify's Template Engine 53% Faster

The Karpathy loop is an AI-assisted optimization technique where an autonomous agent runs hundreds of edit-benchmark-discard cycles overnight, surfacing performance improvements that no single engineer would have the time to find manually — and it just made Shopify's Liquid template engine 53% faster with 61% fewer memory allocations. On March 13, 2026, Tobias Lütke — Shopify's CEO, the person who originally built Liquid over 20 years ago — submitted GitHub PR #2056 against the Liquid codebase with those numbers attached. The method: approximately 120 automated experiment loops using a variant of Andrej Karpathy's autoresearch system. The results: 974 unit tests pass, zero regressions, and a performance improvement visible in production-scale benchmarks against real Shopify themes. Here's how the technique works, how to run it yourself, and what it actually found inside Liquid's parser. What the Karpathy Loop Actually Is Andrej Karpathy open-sourced autoresearch on March 8, 2026 — a de