
How I made a 31M-row SQLite query go from 168 seconds to 5 seconds
My property comparison app was working great in development. Then a user in Norwich searched for nearby sales and my API took 168 seconds to respond. nginx gave up at 60 seconds and returned a 504. Here's how I fixed it -- and what I learned about SQLite performance at scale. The Setup I built a property comparables API that serves recent sales data from government open datasets. The UK version uses HM Land Registry Price Paid Data -- every residential property transaction in England and Wales since 1995. That's 31 million rows in a single SQLite database. 8.7GB on disk. The API is simple: give it a postcode, it finds nearby sales within a radius, filtered by property type and date range. The stack is Python + FastAPI + SQLite, running on a single $20/month VPS. For most postcodes, it was fine. London postcodes with a few hundred results came back in under a second. But Norwich postcode NR1 has 1,941 postcodes in its search radius -- and that's where everything broke. The Query SELECT
Continue reading on Dev.to Python
Opens in a new tab



