I Treat Every AI Answer as a Maybe

When I first started trusting a model with real work, I made a quiet category error. I treated its answers like facts. They came out polished and certain, so I filed them next to things I knew, instead of next to things I’d been told.

A few confident, fluent, completely wrong answers cured me of that. Now I treat every output as a hypothesis — a claim to be tested, not a fact to be pasted. It’s a small reframe with a big effect. A fact you accept. A hypothesis you check.

The important thing I had to internalize is that the model’s tone carries no information about its correctness. It sounds exactly as sure when it’s right as when it’s inventing something. So the confidence in the answer tells me nothing; only my verification does. Run it. Test the edge case. Trace the claim back to a source I trust.

I already do this with people, without thinking. If a smart colleague says “it’s probably this,” I don’t merge their guess unchecked — I take it as a strong lead and verify. The model earns exactly that much trust: a useful, plausible, unproven starting point. Not less, because it’s often right. Not more, because “often” isn’t “always.”

So I take the hypothesis gratefully, and then I do the work to actually know. Trust gets earned one verified claim at a time, never granted wholesale because something sounded sure.

– Serguey Asael Shinder

Leave a comment