Building a Real-Time AI Piano Coach with Gemini Live API

Building a Real-Time AI Piano Coach with Gemini Live API Created for the Gemini Live Agent Challenge hackathon GeminiLiveAgentChallenge The Problem: Piano Practice is Lonely Music practice has always been a solitary endeavor. You sit alone with your instrument, a metronome, and maybe a YouTube tutorial. Even modern piano apps feel like rhythm games -- they grade you on note accuracy but can't see your hands, can't hear your tone quality, and certainly can't have a conversation about your playing. When Google announced the Gemini Live Agent Challenge , I saw the chance to build something different: an AI that doesn't just listen to notes, but truly coaches -- seeing your hands, hearing your music, reading your MIDI data, and talking to you naturally in real time. What is PianoQuest Live? PianoQuest Live is a real-time multimodal AI piano coach powered by the Gemini 2.5 Flash Native Audio model. It processes three simultaneous input streams: Voice and Audio (Microphone): Talk to Gemini n