Designing Asynchronous APIs with a Pending, Processing, and Done Workflow

The Problem Modern backend systems often integrate OCR, machine learning inference, or heavy data processing jobs that cannot complete within a typical HTTP request lifecycle. When a user sends a request that triggers a long-running operation, keeping the HTTP connection open until processing completes is usually a poor design choice. Long-running synchronous requests can still increase the risk of timeouts, tie up resources, and make failure handling more difficult, even when a platform supports longer request durations. Although longer synchronous timeouts are possible in some environments, asynchronous APIs are still valuable as an architectural choice for resilience and better user experience. For example, AWS API Gateway increased its integration timeout limit beyond 29 seconds in June 2024 for Regional and private REST APIs, though with trade-offs such as possible reductions in account-level throttle quota. The Core Idea A practical way to handle this problem is to accept the req