Building VoxAgent: A Local Voice-Controlled AI Agent with Whisper, Ollama, and Safe File Actions

If you ask most AI demos to do something useful, they usually stop right before the interesting part. They can transcribe your speech, explain what they think you meant, and generate a polished response. But they often do not cross the line into safe, visible action on a real machine. For my Mem0 AI/ML & Generative AI Developer Intern assignment, I wanted to build something more practical: a local-first voice-controlled AI agent that could listen to spoken commands, understand user intent, execute local tools, and expose the whole pipeline in a simple UI. That project became VoxAgent . What VoxAgent Does VoxAgent is a local-first AI agent that supports: microphone input uploaded audio files local speech-to-text local intent understanding safe file and folder creation code generation into files text summarization general chat a UI that shows the full pipeline from audio to action The key requirement was not just to generate responses, but to actually perform useful tasks while staying w