by Serguey Shinder
It happened gradually, the way most surrenders do. I started using a model to help me think through a class of decisions, small ones at first, framing questions, comparing options, laying out tradeoffs. It was good at it, genuinely good, and it made me faster. Then one day I noticed I had stopped comparing its options and started simply taking them. Somewhere along the line the tool had crossed from advising me to deciding for me, and I had not been present for the moment it happened.
The decisions in question were not trivial. They were the kind that shape a small piece of a product, the kind I am paid to own. And I had drifted into a habit where I asked the model what to do, read its answer, felt the small relief of not having to hold the whole problem in my head, and acted. I told myself I was still in charge because I could have overruled it. But I never did, and a veto you never exercise is not authority. It is decoration.
What unsettled me was not that the model's decisions were bad. Mostly they were fine. It was that I had lost the thread of why. I could no longer explain the reasoning behind choices I had supposedly made, because I had not made them, I had ratified them, and ratification without understanding is just obedience with extra steps. If someone had asked me to defend one of those decisions, I would have had to go back and ask the model to explain what I had already done.
There is a specific danger in a tool that is right often enough. If it were wrong regularly, I would stay alert, I would check. Because it was usually right, I relaxed, and the relaxation was the whole problem. Competence in the tool had quietly purchased my inattention, and my inattention was the thing that mattered, because on the day its answer was subtly wrong I would have shipped it with the same drowsy trust as all the others.
So I redrew the line, not around what the model was allowed to touch, but around what I was allowed to skip. It can advise on anything. But on the decisions I own, I have to be able to state the reasoning in my own words before I act, without looking back at its answer. If I cannot, then I have not decided, and I am not permitted to pretend that I have.
The tool did not take the decision from me. I handed it over, one comfortable afternoon at a time, and never noticed the weight leaving my hands. Getting it back was not about trusting the model less. It was about refusing to outsource the one thing that was actually mine, which was not the answer, but the understanding of why the answer was right.
– Serguey Asael Shinder
Leave a Reply