The Dual-Write Problem: Why Your Payment API Is One Crash Away From Silent Data Loss

You commit a payment to your database. Then you publish an event to Kafka so downstream services can settle it. Both succeed — until one day the process crashes in the 3 milliseconds between those two operations. The database says the payment happened. Kafka never heard about it. The settlement worker never ran. The customer was charged and nothing moved. That's the dual-write problem. This post explains why it's unsolvable with the obvious approaches, and how the Outbox pattern fixes it properly — using an implementation I built and load-tested to 1,000 concurrent users with zero duplicate charges. Why the Obvious Solutions Don't Work "Just publish to Kafka first, then write to the DB." Same problem, reversed. The event fires but the payment row never gets written. Your downstream consumers process a payment that your database has no record of. "Use a transaction that wraps both." You can't. A database transaction and a Kafka publish are two entirely separate systems. PostgreSQL has n