by Serguey Shinder
Our pipeline took eleven minutes, and with queueing on a busy afternoon it was closer to fifteen. That is not a scandalous number. I would have described it, if asked, as slightly annoying and basically fine.
What I noticed, on an afternoon when a build failed and I could not tell which of my changes had broken it, was that I had five unrelated things in it. A bug fix, a logging change, a rename, a dependency bump and a small piece of the feature I was actually supposed to be doing. I had not decided to batch them. It had simply become obvious, some months earlier, that you do not spend a fifteen minute slot on a one line change, so I had stopped doing that, and I had never noticed the moment I stopped.
Working out which of the five had broken it took me about twenty-five minutes of undoing things by hand, which is the tax that batching charges and always eventually collects.
The larger effect was one I could not see at all while it was happening. The length of that loop had quietly set the smallest change I considered worth making, and once a change has to be worth fifteen minutes, a whole category of work disappears. Renaming something misleading. Deleting a function nobody calls. Splitting a method while I still remember why. None of those survive a fifteen minute toll, so I stopped doing them, and I stopped in a way that produced no visible event and no decision I could have defended or attacked.
What got it fixed was not asking for faster builds, which I had done twice, in the terms everybody uses, about developer time and hours per week. Nobody funds that. What worked was describing the behaviour instead. I showed two months of my own commits, with the five-in-one ones marked, and said that our pipeline had made me a worse engineer in a specific and measurable way, and that I had not agreed to it.
We split the suite. A two minute run on every push covering the fast tests and the build itself, the full run before a merge and again nightly. Most of the eleven minutes turned out to be a dependency download we were doing from scratch every time, which somebody fixed in an afternoon once it was anybody's job.
I make small changes again. That is the whole return, and it is worth more than the minutes.
The thing I would tell anyone is that you will never experience a slow loop as a change in what you do. You will experience it as a preference for larger pieces of work, and you will think it is your taste.
– Serguey Asael Shinder
Leave a Reply