Serguey Asael Shinder – IT Professional

by Serguey Shinder

I asked a model to summarise a long technical document I did not have the patience to read myself. It gave me back a clean, confident summary, and I felt the small relief of a chore removed. I forwarded a decision based on that summary, and only a week later, when the details actually mattered, did I discover that one of the key points had been softened into something the original document never said.

It was not a hallucination, exactly. It was worse in a subtler way. The summary had smoothed a hedge into a certainty. The document said a change might reduce load under certain conditions. The summary said the change reduces load. Between might and does lies most of engineering, and the model had quietly stepped across that line to give me something cleaner to read.

What unsettled me was my own role in it. I had not been fooled so much as I had wanted to be. Reading the real document was tedious. The summary was easy, and it arrived wearing the confident tone we associate with someone who has done the reading. I outsourced not just the effort but the caution that effort would have forced on me.

I have come to think this is the actual risk of these tools in my work. Not that they are wrong, but that they are pleasant to agree with. A confident, fluent paragraph lowers your guard in a way a messy human email never does. The friction that used to protect me, the difficulty of getting a clear answer, was doing quiet work I did not appreciate until it was gone.

Now I use these tools differently. I still ask for summaries, but I treat them as a map, not the territory. When the map points somewhere a decision will actually depend on, I go and walk that ground myself. The summary tells me where to look. It does not get to tell me what I found.

I have also started asking the model to show me what it is unsure about, to surface the hedges rather than smooth them away. The results are less satisfying to read, which is precisely why I trust them more. A little visible doubt is a feature, not a defect.

The strange thing is that this has made me a more careful reader in general. The ease of machine-generated confidence taught me, by contrast, how much of my own understanding used to come from struggling through a hard text. I do not want to lose that struggle entirely. Some things are supposed to be earned, and a summary, however good, is not the same as having read the thing.

– Serguey Asael Shinder