by Serguey Shinder
In the spring of 2025 we took on about three thousand new product lines from four suppliers, and their spreadsheets were not written for customers. Columns of codes, dimensions, finishes, and very few sentences. We used a model to turn each row into a product description, and a member of the catalogue team read a sample of them for tone and spelling. They were good. They read better than the descriptions people had written.
In July a joiner rang our Bristol branch about an internal door blank he had ordered for a block of flats. The product page said it was an FD30 fire door, thirty minutes of resistance, with the relevant test standard named. The label on the door said nothing of the kind. He had noticed because fire doors carry a certification mark, and this one did not.
The supplier's spreadsheet had a column for fire rating, and on that line it was empty. Nearly every other door in the same range was rated, and the model, asked to write a complete and helpful description, had written the description the neighbouring rows suggested. It had not misread anything. It had filled a gap with the most likely value, which is what it is built to do, and in a fire door the most likely value is a claim about whether people get out of a building.
We traced every order of that door. There were two, and neither had been fitted. Then we checked the rest. Among the three thousand descriptions, eleven stated a performance figure, a standard or a certification that did not appear anywhere in the supplier's data. A load rating on a bracket. An IP rating on an outdoor socket. Our sample check had read for whether the text was well written, and every one of the eleven was well written.
We changed what the model is allowed to say, and how we check it. It still writes descriptions, but it is told explicitly which attributes are blank and that blank means unknown. Every generated description then goes through a check that pulls out each number, unit, standard reference and certification term, and looks for each one in the supplier's fields. Anything that cannot be matched holds the description back for a person. And for fire, structural and electrical products, the performance claims are not written by the model at all. They are inserted from the data by a template, and if the field is empty, the page says the information has not been supplied.
The model writes the sentences around the claims now, and the claims come from a field with a supplier's name against it. Fluency is a fine thing to buy. Facts that someone can be held to were never something a model could sell us.
– Serguey Asael Shinder
Leave a Reply