How to do Regression Testing for MCP Servers

If you maintain an MCP server, there is a class of breakage that no amount of unit testing will catch. Someone on your team renames a tool parameter from query to search_query , or rephrases a tool description from "Search the web" to "Search the web for recent results," and the change passes every test in the suite because nothing actually validates the protocol surface your server exposes to AI agents. These schema drift issues are not always visible outright, but they accumulate over time and tend to surface as baffling agent failures — tools that stop being selected, arguments that arrive malformed, responses that get misinterpreted — precisely because MCP tool descriptions are not documentation in the traditional sense. They are instructions. The model reads them to decide when to call a tool, how to invoke it, and what to do with the result. A reworded description is not a cosmetic change. It is a change in the instruction set the model operates from. The ecosystem has clearly st