Building AI Video Transcription with OpenAI Whisper

I built a video transcription feature into my side project — a free video downloader called Videolyti . Here's how I wired up OpenAI's Whisper model to transcribe downloaded videos on the server side, what worked, what didn't, and what I'd do differently. Why Server-Side Transcription? Most transcription tools either charge per minute of audio or require you to upload files to some third-party API. I wanted something that runs on my own hardware, costs nothing per request, and integrates directly with the download pipeline. OpenAI Whisper was the obvious choice. It's open source, handles 90+ languages, and the accuracy on the large-v3 model is genuinely impressive — even with background noise and accented speech. The Architecture The stack is straightforward: Express 5 backend with Socket.IO for real-time progress updates yt-dlp handles video downloading from YouTube, TikTok, Instagram, etc. ffprobe extracts audio duration metadata Whisper CLI runs the actual transcription The flow: us