by Serguey Shinder
In February we adopted a tool that writes commit messages from the diff. By July it had written about two thousand three hundred of them, and every one read better than what we used to produce ourselves. Full sentences, correct tense, a summary line that fitted, and a paragraph underneath explaining what the change did.
In July we spent half a day finding out why our nightly reconciliation had started ignoring anything older than ninety days. Customers with long running accounts were seeing corrections vanish, and nobody on the team could say when the behaviour had arrived or on whose authority. The history led us straight to a commit in March, which was the system working. The message said, fluently, that it introduced an age filter on the reconciliation query with a configurable threshold defaulting to ninety days.
That is true. It is also precisely and only what the four lines of code already said, in more words. The author had left us in May.
Three of us then read code and guessed for most of a morning, and in the end the answer was in a customer support thread from the previous February, where an account manager had explained that a particular client's back dated corrections kept reopening months that finance had already closed. Somebody had gone away and implemented that, correctly, and the reason for it had never entered our systems at all.
The thing I had not thought through is structural rather than a complaint about quality. A message generated from a change can contain nothing the change does not already contain. It is a second copy of the diff in English, and its fluency is what makes it feel like a record. Our old messages were often terrible, but they were terrible in a leaky way. Somebody would write skip the old ones, per the thing Marie raised about closed months, and that half sentence is worth more than any correct paragraph, because it points outside the repository to where the reason lived.
We stopped asking for commit prose entirely, which surprised people. The tool still writes it and nobody reads it. What we added instead is one required line on the pull request, which asks what happened outside this codebase that made this change necessary, with a link if a link exists. A ticket, an email, a regulation, a customer's name, a conversation in a corridor. About a fifth of the time the honest answer is that somebody just wanted it tidier, which is a perfectly good thing to have written down.
The question I now ask of anything a machine produces for us is which part of this existed only in a person's head. Whatever that part is, the machine cannot see it, and if the artefact it produces looks complete without it, we will stop noticing it is missing until the day we need it.
– Serguey Asael Shinder
Leave a Reply