The Spatial Eye: Bridging the Physical World with Gemini 2.5

Disclaimer: Created for the purposes of entering the Gemini Live Agent Challenge. What happens when an AI assistant doesn't just listen... Disclaimer: Created for the purposes of entering the Gemini Live Agent Challenge. (Note: generated with nanobanana) What happens when an AI assistant doesn't just listen to your voice, but actually sees exactly what you see, continuously, in real-time? For the Gemini Live Agent Challenge , we built The Spatial Eye —an open-source, multimodal AI assistant designed to bridge the gap between complex physical environments and agentic AI. In this post, we'll dive into the architecture of the application, how we used the new Gemini 2.5 Multimodal Live API via the Agent Development Kit (ADK), and how we deployed a unified architecture to Google Cloud Run. The Problem: Information Asymmetry In specialized settings—whether it's mechanical repairs, a surgical setup, or debugging a complex hardware configuration—users often struggle to match an AI's text or ve