From 80-Second APIs to Sub-Second: Rebuilding a Geospatial Backend with Async Pipelines

From 80-Second APIs to Sub-Second: Fixing Latency with Async Pipelines (Django + Celery) Introduction At some point, every backend engineer hits this wall: The API works perfectly… until it doesn’t. I hit that wall with a farm analytics endpoint computing NDVI (Normalized Difference Vegetation Index) from satellite imagery. The system was correct, the logic was sound, and the results were accurate. But the numbers told a different story: P95 latency: 1.25 minutes That’s not an API. That’s a blocking compute job pretending to be one. This is the story of how I redesigned the system—from a synchronous request-driven model to an asynchronous data pipeline—and brought latency down to sub-second performance (P95 ≈ 725ms) . The Original Architecture (The Hidden Problem) At first glance, the system looked clean: [Client] ↓ [Django API] ↓ [STAC API → Satellite Data] ↓ [Raster Processing (NDVI)] ↓ [Response] What happened on each request? Query satellite imagery via STAC Fetch raster bands (Red