Solving Audio Gaps in Real-Time Speech Translation

When building a real-time Speech-to-Speech (S2S) translation service, latency is usually the enemy everyone talks about. But there's a silent killer (quite literally) that can ruin the user experience just as effectively: audio gaps. In our journey migrating from Flask to FastAPI and implementing Nvidia Riva, we encountered a persistent issue where our synthesized audio had audible stuttering—specifically, 20ms gaps of silence between chunks. Here’s how we diagnosed and fixed it, turning a robotic output into a smooth, natural conversation. The Problem: "Machine Gun" Audio Our pipeline looked standard: Receive user audio (WebSocket) Transcribe (ASR) & Translate (NMT) Synthesize speech (TTS) Stream audio back to the client But the output sounded like a machine gun. Words were clear, but the flow was choppy. Opening the raw audio dump in Audacity revealed the culprit: consistent 20-50ms gaps of silence inserted between every audio chunk returned by the TTS service. What Wasn't The Cause