FTS vs Hybrid Memory Search: A Real-World Benchmark

TL;DR Mode Exact Keywords Paraphrased Contextual Overall FTS-only 85% 30% 30% 48% Hybrid 85% 55% 55% 65% FTS-only is surprisingly capable for direct lookups. But the moment users rephrase or ask abstract questions, hybrid search pulls ahead by +25 percentage points . Why This Matters SoulClaw's 4-Tier Memory system stores everything an AI agent learns — from daily conversation logs to long-term decisions and project context. When your agent needs to recall something, the retrieval method determines whether it finds the right memory or returns noise. Most hosted AI agents use one of three approaches: Full-Text Search (FTS) : Keyword matching. Fast, free, requires no ML model. Semantic Search : Vector embeddings (e.g., bge-m3 via Ollama). Finds conceptually similar content even without exact keyword overlap. Hybrid : Combines both, re-ranks results. SoulClaw's default when an embedding provider is available. We wanted hard numbers on the actual difference — not synthetic benchmarks, but