How I Built Niobe: An AI Waitress with Gemini Live and Google Cloud

Disclaimer: I created this blog post for the purposes of entering the Gemini Live Agent Challenge hackathon. When sharing on social media, use the hashtag #GeminiLiveAgentChallenge. What is Niobe? Niobe is an AI waitress for restaurants. Restaurant owners upload their menu (even as a photo), get a shareable link, and customers can talk to the waitress by voice—ask about the menu, place orders, and have a natural conversation. No app install required; everything runs in the browser. I built it using Google’s Gemini (including the Gemini Live API for real-time voice) and Google Cloud for deployment. Here’s how I put it together. Two places where Gemini powers the product I use Gemini in two different ways: 1. Gemini API: turning menu images into structured data Restaurant owners don’t have to type their menu. They upload images (photos of the menu, PDFs, etc.). The Laravel backend sends those images to the Gemini API (via the Laravel AI package with the Gemini driver). Gemini returns str