How I Built SilentEar — A Real-Time AI Accessibility Agent for Deaf Users with Gemini Live API

This blog post was created for the purposes of entering the Gemini Live Agent Challenge hackathon. #GeminiLiveAgentChallenge The Inspiration — Why This Is Personal This project is deeply personal to me. My son is profoundly deaf by birth, and as he began learning Pakistan Sign Language (PSL) at school, I joined basic training to learn alongside him. During this ongoing journey, I observed firsthand how challenging daily life can be for the Deaf community and their families. Often, deaf individuals feel irrelevant or isolated in gatherings, struggle to communicate smoothly with hearing people, or face physical danger because they cannot hear critical environmental alerts. While we've seen incredible advancements in AI, accessibility tools have largely stagnated around simple speech-to-text transcription. I realized that transcription misses the most crucial part of environmental awareness: the context . A fire alarm, a knock on the door, or a baby crying are critical sounds that text on