by Serguey Shinder
I spent most of 2019 arguing for a rewrite of the notifications service. It was old, it was slow, and I was certain that the missed notifications people complained about were coming out of it. I won the argument, we spent about four months on the work, and in the quarter after it shipped the support tickets in that category fell by roughly two thirds. That was the end of the matter. Somebody said well done in a meeting, the number appeared in a slide, and I carried the conclusion out of the building with me.
The conclusion was that I had read the situation correctly when other people had not, and I used it. Twice in the years afterwards I argued for a rewrite on the strength of it, in rooms where anybody who disagreed had to argue against a thing that had demonstrably worked.
I found out it was not true on a Tuesday afternoon four years later, going through old tickets for an entirely unrelated reason. In the same month our rewrite went live, someone in support had rewritten the wording of the notification email and made one field on a form optional. The tickets that stopped arriving were not about notifications failing to arrive. They were about a message nobody could understand, sent because people were filling in a form wrongly, and she had dealt with both in an afternoon that appears in no plan I have ever seen.
The rewrite may well have been worth doing on its own merits. What it plainly did not do is the thing I had been telling people it did for four years.
Nothing in the way we work was ever going to catch that. When something fails we stop, write a document, gather everyone, establish a sequence of events and argue about the cause, and we do all of it because a failure demands an explanation from somebody. When something works, the explanation is whatever was most visible, claimed by whoever was most invested in it, and nobody asks for a timeline, because there is no discomfort in the room to resolve. Success is the only outcome we accept without any evidence at all.
Two things came out of that afternoon and I have kept both. Before anything large, I write down the specific number I expect to move and by how much, somewhere I will find it later, because a prediction produced afterwards is not a prediction. And when something improves, I spend twenty minutes establishing what else changed in the same fortnight, which nearly always turns up a colleague quietly doing something sensible that nobody recorded.
I am considerably more suspicious of the things I got right than of the things I got wrong. The wrong ones were investigated by people with an interest in accuracy. The right ones were explained by me.
– Serguey Asael Shinder
Leave a Reply