How I Built a System to Detect When LLMs Don't Know Something

How I Built a System to Detect When LLMs Don't Know Something The Problem I Was Trying to Solve I've been deploying LLMs to production for a while, and kept running into the same issue: the model's confidence scores don't tell you when it lacks knowledge . For example, an LLM might give you "90% confidence" for both: "What is the capital of France?" (factual) "What will the stock market do tomorrow?" (impossible to know) The confidence score is the same, but one is safe to trust and the other isn't. Understanding Two Types of Uncertainty After researching uncertainty quantification, I learned there are two fundamental types: Aleatory Uncertainty (Q1): Inherent randomness in the data Example: "Will this coin flip be heads?" Irreducible - no amount of data will eliminate it Epistemic Uncertainty (Q2): Uncertainty due to lack of knowledge Example: "What happened on Mars yesterday?" Reducible - could be resolved with more data The key insight: standard LLM confidence scores mix these toget