by Serguey Shinder
There was a test I trusted for the better part of a year. It sat green in every run, guarding a payment calculation I considered one of the more important things I had written, and every time I saw its little tick I felt a quiet reassurance that this part, at least, was safe. I never opened it, because you do not open the tests that pass. You open the ones that fail.
The truth came out during an unrelated change, when I finally had a reason to read the thing. The test asserted that a function returned a number. It did not check which number. Somewhere in a refactor long ago, the specific expected value had been deleted and replaced with a check that the result was merely present, and the test had gone on passing ever since, cheerfully confirming almost nothing.
For a year it had guarded a door that was already open. The calculation could have drifted, could have started returning subtly wrong totals, and this test would have stayed green through all of it, because green was all it knew how to be. It was not testing correctness. It was testing existence, and existence is a very low bar for a number to clear.
What unsettled me was not the gap itself but how safe the gap had felt. A missing test announces itself eventually. You feel its absence when something breaks unwatched. But a test that passes for the wrong reason is worse, because it actively radiates confidence. It is a green light wired to nothing, and you steer by it precisely because it looks like it is working.
I fixed the assertion, watched it fail against a deliberately broken input the way it always should have, and only then trusted it. That failure, the first honest thing it had done in a year, was the most reassuring red I have ever seen. A test that cannot fail is not protecting you. It is decorating your test suite.
Since then I have made a small ritual of it. When I write a test, I break the code on purpose to make sure the test notices. If it stays green while the thing it guards is visibly wrong, the test is a lie, however good it looks in the summary. I would rather have no test than a test that tells me everything is fine while quietly checking nothing.
The lesson has outlived that one function. I have learned to distrust reassurance that I have never once seen earn its keep. A safeguard I have never watched do its job is not a safeguard. It is a story I am telling myself, dressed up in the color green.
– Serguey Asael Shinder
Leave a Reply