Building Aura: A Multimodal Smart Home Operated by Gemini Live 🌌

💡 The Problem with Smart Homes Smart homes today are often fragmented and reactive. You speak into a puck on the wall, and it toggles a light on a screen. There is no continuous awareness. For the Gemini Live Agent Challenge 2026 , I wanted to build something that feels alive . Inspired by futuristic sci-fi interfaces, I built Aura — a central AI operating pilot that doesn't just hear you, but sees your environment concurrently and translates that intelligence into a living, responsive Ambient Dashboard layout natively. 🚀 What is Aura? Aura is a fully multimodal smart home operating system utilizing bidirectional WebSockets over continuous, low-latency backpressure limits. Unlike previous generations of voice assistants that rely on turn-taking (Speech-to-Text ➔ LLM ➔ Text-to-Speech), Aura streams continuous raw audio and webcam frames concurrently using the google/genai Node SDK. 🛠️ The Architecture I engineered a decoupled reactive container pipeline deployed on Google Cloud Run : ⚡