most AI-generated tests are worse than no tests

i started having claude write tests for my project and quickly realized something: most of them were useless. they passed, but they didn't test anything and i had a major sense of false security seeing 12/12 tests pass, etc. over and over. a test that asserts expect(result).toBeDefined() after calling a function is technically a passing test. it will never fail unless the function throws. this was like 80% of what i was getting. tests that exercised code paths without actually checking that the code did the right thing so I had great code coverage but stuff was still breaking constantly. so i started thinking about what makes a test actually worth having, and i ended up with a set of gates that changed how i think about it. the mutation test is the most important one. take a passing test, go flip a condition or change a return value in the source code, and run the test again. if it still passes, then the test is garbage because it's not actually sensitive to the behavior it claims to t