by Serguey Shinder
When I first started leaning on a model for hard problems, I treated it like a vending machine for answers. I would type the problem, take the response, and move on, and for simple things this worked well enough that I did not think to do anything else. The model gave answers, I wanted answers, and the transaction seemed complete. It took a while, and a few expensive mistakes, to understand what I was skipping.
The mistakes had a pattern. They were never cases where the model gave a wrong answer to my question. They were cases where it gave a right answer to a question that was subtly not the one I actually needed answered. I had asked something slightly imprecise, something that made sense in my head but underspecified the real problem, and the model, obligingly, had answered exactly what I asked rather than what I meant. The failure was in my question, and the answer had faithfully inherited it.
What I slowly learned was to spend my effort on the question rather than the answer. Before I ask the model anything now, I ask myself what I am actually trying to find out, what a good answer would even look like, what I would do differently depending on which way the answer went. Half the time, doing that reveals that my question was vague, or that I was asking about a symptom rather than the thing beneath it, or that I did not actually need the answer at all.
This changed the whole relationship. The model is extraordinary at answering, which means the quality of what I get out of it is almost entirely determined by the quality of what I put in, and the quality of what I put in is a thing only I can improve. A sharp question to a capable model produces something genuinely useful. A fuzzy question to the same model produces a confident, fluent, well-structured response to the wrong thing, and the fluency makes the wrongness harder to spot.
The discipline is not really about the tool at all. It is about resisting the pull of the easy answer long enough to make sure I am asking the right question, which is the part that was always mine to do and that no model can do for me, because it depends on knowing what I actually need, and I am the only one who has access to that.
So the most valuable thing the tool taught me was not any of its answers. It was the habit of interrogating my own questions first, of treating a fuzzy question as a problem to be solved before it is a thing to be asked. The answer was never the hard part. The hard part was knowing what to ask, and that is still, entirely, my job.
– Serguey Asael Shinder
Leave a Reply