by Serguey Shinder
We had a logging habit that felt responsible. When something failed, we logged everything we had, the full request, the full response, every field, so that whoever debugged it later would not be starved for context. It felt like diligence. It was, for a long time, invisible, because logs are the one place in a system nobody looks until something is already wrong.
The reckoning came during a routine review of where our logs actually went. They flowed into a service several teams could read, retained for months, searchable by anyone with access. And in those searchable, long-retained, widely-read logs sat things that should never have travelled that far. Full payloads with personal details. Tokens captured mid-flight. The occasional password that a client had put in the wrong field and our helpful logger had faithfully recorded.
Nobody had decided to build a vast searchable archive of secrets. We had decided to be helpful when things broke, and the archive was the accidental sum of that helpfulness repeated a thousand times a day. Every generous log line had been a tiny leak, and the leaks had pooled into something that would have been a genuine catastrophe in the wrong hands.
What unsettled me most was how good our intentions had been. This was not carelessness. It was care pointed in the wrong direction. We had optimised for the comfort of the future debugger and forgotten that a log is not a private note to a colleague. It is data, with a lifetime and a blast radius, sitting somewhere long after the incident that created it is forgotten.
We spent weeks after that redacting at the source, deciding field by field what was safe to record and what had to be masked before it ever reached disk. The tedious part was not the code. It was admitting how much we had been storing without ever asking whether we should, simply because storing it had never felt like an action.
I have carried one rule out of that experience. A log line is a decision about what to remember, and remembering is not free or neutral. Every field I write down is a small promise that it is safe to keep, safe to search, safe for anyone with access to read at leisure. Most of what I used to log did not deserve that promise.
I still want the future debugger to have what they need. But I no longer confuse generosity with responsibility. The most responsible log is often the one that deliberately says less, because the thing you never wrote down is the only thing that can never leak.
– Serguey Asael Shinder
Leave a Reply