From RAG to a “memory layer”: what building an AI assistant taught us

About a year and a half ago, we were building a proactive AI assistant. Not just a chatbot, but something that could actually act on your behalf. It could reply to emails in your tone, move calendar events, organize your inbox, and surface information based on what you actually care about. The goal was simple: build something that feels like an extension of how you think. The part we didn’t expect To make that work, we started with what most people use today: RAG. And to be fair - RAG works. You can go pretty far with chunking, embeddings, and retrieval. You can build systems that feel smart. But as the assistant got more complex, something started to break. Not in an obvious way. It was more subtle. The system could retrieve relevant information, but it didn’t really understand how things were connected. Everything was based on similarity. And similarity is not structure. Building a "brain" To move forward, we needed something else. We started building what we internally called a "bra