by Serguey Shinder
We put the help centre through a model in June and had German, French, Spanish and Italian by the Thursday. Two hundred and forty articles. It cost roughly what one professional translator would have charged for a fortnight, it read beautifully, and everyone who looked at it said so, including two colleagues who speak French and said the tone was right.
The mistake was one sentence in the cancellation article. Our English says a cancellation takes effect at the end of the billing period already paid for. The German said something a good deal closer to immediately, with the remainder refunded. It was a perfectly fluent sentence. It was also a promise we do not make and cannot afford to make, and about ninety customers cancelled on the strength of it over five weeks, until a support agent in Berlin asked why so many of them were surprised by their final invoice.
We honoured every one, which was the only decent option and cost a little under nine thousand euros.
What I got wrong was not trusting the model. The translation was, by any reasonable measure, good. What I got wrong was designing a review step that could not possibly have caught this. We asked people to check that the text read well, because reading well is something a non speaker can half assess by watching a fluent colleague nod at it. Nobody was asked whether the German sentence and the English sentence made the same commitment, and the two people who could have answered that were never shown it, because the work already looked finished.
That is the general shape, and I keep meeting it. Machine output fails hardest exactly where you have no ability to judge it, which is the same reason you asked for it. In the areas where I can evaluate an answer I did not need much help. In the areas where I needed help I have no way of knowing, and fluency reads as correctness to everybody who cannot check.
So the rule we work to now is narrow. Anything stating an obligation, a price, a deadline or a right is read by a person who speaks the language and knows what we actually do, before it goes anywhere near a customer. That was fourteen articles out of two hundred and forty and it took one day. Everything else is machine translated and corrected as people complain, which is an honest enough arrangement for an article about changing your avatar.
Before I accept anything I cannot assess myself, I now ask who could tell me it was wrong, and whether that person has been shown it. If the answer to the second half is no, what I am holding is not verified work. It is fluent work, which is a different thing entirely, and the difference shows up in the invoices.
– Serguey Asael Shinder
Leave a Reply