Changelog — 2.163–2.170

A flaw in this morning’s fix, found before it ever ran

• The change we made earlier today — having the audio system get itself ready on startup — had a gap in it. It checked once whether the part it depends on had finished starting, and if the answer was “not yet”, it gave up rather than waiting. It now waits.
• It would almost always have got away with it, which is precisely what would have made the occasional failure so hard to understand: everything would look normal, and once in a while the system would sit there not ready, for reasons no message would explain.
• It is the same mistake we made twice in one day — treating “hasn’t happened yet” as “isn’t going to happen”. Both times it only misbehaved in the few seconds after an update, which is when nobody is looking closely and everything is expected to be a bit unsettled.
• Nothing to do at your end — this was caught before it reached you.

After an update, the audio system was “on” without being ready

• Yesterday’s fix worked, and doing the update revealed the next thing. When the part of the system that reads text aloud is updated, it comes back knowing none of its voices — and it only goes and fetches them when somebody actually asks for audio. So for a while after every update it was running, reporting itself healthy, and would have failed the first request made of it.
• It now gets itself ready as soon as it starts, instead of waiting to be asked. Nobody has to remember to go and poke it, and the first person to want audio is no longer the one who discovers whether it works.
• The interesting part is why it isn’t done the obvious way. Making it get ready before declaring itself running would seem safer — but it can take minutes to load, and the thing that watches it would have concluded it had hung and restarted it. Repeatedly. A safety improvement that causes an outage is worse than the problem. So it now starts, says it is running, and gets ready in the background.
• And we fixed our own check, which had started crying wolf. The check we added yesterday treated “started a moment ago and not ready yet” as a fault, so a perfectly healthy system looked broken every single time it was updated. A check that is wrong in a routine situation gets ignored within about three weeks, and then it is worth nothing when it is right.
• Nothing to do at your end.

A routine update stopped anything being read aloud, and nothing noticed

• For an evening last week, nothing could be turned into audio. The part of the system that produces spoken narration keeps its list of voices in memory rather than on disk, so when it was updated it came back knowing none of them. Every request for audio failed, and it stayed that way until someone tried to use it.
• The reason nobody was told is the part worth fixing. Everything that watches this said it was fine — the health checks, the status readouts, the routine after-update verification, all green, all correctly reporting what they were built to look at. None of them was looking at whether the voices were actually there. It is a strange feeling to read a full page of ticks next to something that plainly does not work.
• It now repairs itself. The system remembers which copy of the audio component it prepared, and notices when that copy has been replaced — whether it was updated or merely restarted, since both lose the voices. The next request rebuilds the list rather than failing. It takes a fraction of a second and nobody sees it happen.
• And there is now a check that can actually go red for it, which is the more useful half. It asks the audio component directly what it is holding, rather than asking through the layer that would quietly fix the problem while answering — a check that repairs what it is inspecting will always tell you everything is fine.
• We also found the same blind spot one step along. If restoring the voice list failed part-way, that looked exactly like it having succeeded, and nothing ever tried again. Those are now two separate facts rather than one.
• Nothing to do at your end.

Seven of the narration voices we offered did not exist

• The list of voices you can pick from when having something read aloud had seven entries that could never produce a sound. Choosing one did not warn you or fall back to something similar — it simply failed. They have been removed. The voices that remain all work, and that is now checked rather than assumed.
• They were left behind by a change made in May. Those seven were a different kind of voice: described in words rather than recorded from a real one. We stopped supporting that kind, correctly — nothing was using it. What we did not do was take them out of the menu, so for three months the menu offered seven things that had quietly stopped being possible.
• Nobody noticed because the failure said the wrong thing. All you saw was that narration had failed. The one useful message — naming the voice and listing the ones that do work — existed, but it could never be reached for these seven, because they failed at an earlier step than the one that produces it. A good error message is no use if the failure happens before it.
• The uncomfortable part, and the reason we are writing it up rather than just fixing it. The note explaining May’s change said these voices were unreachable from anywhere in the site. They were not — they were in three separate menus. The decision it justified was still the right one, so nobody re-read the reasoning. A conclusion being correct is not evidence that the reason given for it was.
• What stops it recurring is a comparison that had never been made. The list of voices we offer and the list of voices that can actually be produced were two separate lists, and nothing had ever put them side by side. Something now does, and it refuses to trust its own clean result unless it can first be shown failing on purpose — a check that has never been seen to fail tells you nothing when it passes.
• Nothing to do at your end.

Some pages were being read aloud from an older version of themselves

• Several pages on the site can be listened to rather than read. When we change one of those pages, the recording has to be made again — otherwise the narration is describing the version before yours, which is worse than having no narration at all. You would have no way of knowing.
• There was a screen for keeping track of this, and it was quietly useless. It marked pages as “outdated” based on something that had nothing to do with whether the page had changed, so in practice it told us everything was fine, always. The page most obviously affected was this one — the changelog gains something every single release.
• It now actually checks. It re-reads each page and compares it against what was recorded, on request rather than constantly — which is what the old version was trying to avoid, and the reason it settled for a marker that could not mean anything. Asking took under two seconds and found six pages needing a fresh recording.
• There is a difference between “fine” and “nobody has looked”, and the screen now says which. If a page fails to load while being checked it says so rather than passing it as up to date. Not knowing something is a different answer from everything being well, and a screen that cannot tell you them apart will always give you the comfortable one.
• And re-recording is now part of the written release routine rather than something someone remembered. It had never been written down, and the reason turns out to be the bug above: nobody writes down a step that a dashboard appears to be handling.
• Nothing to do at your end — the affected recordings are being remade.

Subtitles now come in Traditional Chinese as well as Simplified

• 繁體中文 has been added, and it is listed separately from 中文 on purpose. A reader who uses Traditional characters can often work out Simplified, and the other way round — but not reliably, and not comfortably at the speed subtitles move. It serves Taiwan, Hong Kong, Macau and a good part of the diaspora.
• This is the first language added because someone could see it was missing. The watching videos page has a link for exactly that, and it now says sixteen rather than fifteen. Existing videos pick the new language up on their own.
• A small thing we found while adding it, which is the useful part. The list of language names existed in two places, and only one of them was the one we had agreed to keep. Adding a single row meant adding it twice — which is precisely how such lists drift apart, and precisely why the moment you have to edit both is the moment to merge them. Now there is one.

Rebuilding a video could quietly bring an archived story back to life

• Asking to rebuild the video for an archived Vignette moved it back to drafts. Nobody was told. An editorial decision to retire something was reversed by pressing a button about video.
• It is the same fault we fixed for Reflections yesterday — saving should never change whether something is published as a side effect — and we went looking for it in the other editors specifically because of that. It turned up in yesterday’s own fix.
• Worth saying why, because it will happen again to someone. Yesterday’s problem was about published items, so yesterday’s fix reasoned about published items, and the third possibility — archived — was simply never in view. A fix inherits the shape of the bug it is fixing.
• Events, Sessions and Articles were all checked and are fine, each for a different and slightly boring reason. Written down so nobody has to check again.
• Nothing to do at your end.

A safety measure we had written down, explained at length, and never actually switched on

• The software that produces our narration had a known rough edge, and we had a setting in place to avoid it. The setting was carefully chosen, carefully explained, and had been quietly doing nothing for months — it was copied forward from an older machine onto newer software that no longer recognises it. When the rough edge finally bit, it was the very thing we thought we had guarded against.
• The reason nobody noticed is the part worth telling, and it is not that nobody noticed. The software has been printing a warning about that exact setting every time it starts, and we had even written the discovery down at the time. What never happened was the configuration file changing. A warning in a log nobody reads is the same as no warning at all.
• Meanwhile our after-deploy check reported everything healthy — five layers of it, minutes after the failure. Not one of them asked which version of the software was actually running, so a completely green report had been compatible with a completely dead precaution the whole time.
• So the new check is not “is that one setting working?” It asks the software to list every setting it knows about, and compares that against every setting we have asked for. Anything we are asking for that it has never heard of now shows up as a failure. That catches the next one without anyone having to remember this one.
• And the underlying cause is fixed rather than papered over. Narration happens in two steps, and the first was allowed to run ten pieces of work at once while the second could only ever handle one. The two now match. It was never a case of “no limit” — it was a limit set to the wrong number, which is a good deal harder to spot.
• Nothing to do at your end.

The machine that makes our voices can only do one thing at a time, and now we act like it

• Yesterday’s outage had a simple shape. Turning text into speech is one job on one machine. Nothing was counting how many of those jobs we asked for at once, so when a couple of video rebuilds and a page narration all arrived together, the thing doing the work stopped. Narration was unavailable until someone restarted it by hand.
• There has been a queue for this all along — it simply did not cover the voices. Picture generation has waited its turn politely for months. Everything to do with speech went straight to the front, every time, because it was never added. The fix was mostly plumbing rather than invention, which is the good kind of fix and the easy kind to never get round to.
• Now every piece of that work takes a ticket. If something is already running, yours waits, tells you what it is waiting for and how many are ahead, and starts the moment the machine is free. We measured it: two narrations asked for at the same instant, and the second began 0.6 seconds after the first ended rather than climbing on top of it.
• Pictures and voices no longer block each other, which they briefly would have under the obvious version of this fix. They are different machines, and making one wait for the other would have been slower for no reason at all.
• Long renders now survive you closing the tab. Video rebuilds used to live inside the page that started them — navigate away and the work went with it. They are proper background jobs now, so you can leave, come back, and watch where it got to. This is also what makes waiting in a queue bearable rather than a browser tab held hostage.
• One honest consequence: a page narration may now sit behind a video rebuild for a while. That is correct rather than a regression — it is one machine and the work genuinely is one-at-a-time — and it is much better than the alternative, which we watched happen.
• Nothing to do at your end.

Editing an old Reflection could quietly take it off the site

• Correcting a typo in a Reflection from March marked it as “scheduled” — a finished piece, filed as though it were still to come. The editor’s main button wrote whatever it was labelled, without ever looking at the dates or at what the Reflection already was.
• That is not just an odd label. Which Reflections appear on the site is decided by that same status, so a small edit could remove a published Reflection from public view until an overnight job happened to put it back. Nobody would have been told, and the edit would have looked like it worked.
• The new rule is simple: saving never changes whether something is published. It only ever promotes a draft. Anything already published, scheduled or finished keeps exactly the state it had, so tidying the wording of an old piece does nothing but tidy the wording.
• And a draft now becomes what it actually is — still to come, running now, or already finished — rather than always “scheduled”. Writing up something that happened last week is a normal thing to do, and it is now recorded as such.
• The button says what it will do. It reads “Schedule” only when the date is genuinely in the future, and “Save” otherwise. The old label agreed with the mistake it was about to make, which is most of why it was easy to click.
• Nothing to do at your end — and if you spot an older Reflection missing from the archive, it should reappear on its own.

The subtitle language menu sat awkwardly under its own button

• On the home page the Captions button, the language menu and the note beneath them stacked into three separate lines where there was comfortably room for two. Tidied.
• The cause was quietly interesting — a drop-down takes its width from its longest entry, whichever one that happens to be, so the row was being pushed over the edge by “Bahasa Indonesia”. That meant it looked fine on some videos and not others, depending purely on which languages each one carried.
• Nothing to do at your end.

A button that did its job and looked like it had failed

• Asking to rebuild a published story’s video sent you straight back to the list, with no message and nothing to watch. The rebuild was running perfectly well — the page that would have shown you was simply gone.
• The interesting part is what happened next. With nothing on screen to say otherwise, the reasonable assumption is that it did not work, so you go back and press it again. Now two rebuilds are running at once. Both need the machine that turns text into speech, and it turned out nothing anywhere was limiting how much work that machine could be asked to do at the same time. It fell over.
• So a missing message became an outage. We would not have predicted that chain, and it is a good reminder that “only a bit of polish” is a judgement about the interface, not about the consequences.
• Fixed, and fixed carefully. The quick version of this fix would have quietly turned a published story back into a draft — taking it off the site because someone pressed a rebuild button. Instead the page is now told explicitly to stay put, and what is published stays published.
• Reflections were checked and never had this problem, for a small reason worth recording: their editor skips the save entirely when the item already exists, so there was never a redirect to inherit.
• Nothing to do at your end.

Our videos now carry subtitles in fifteen languages

• Every Reflection and Vignette made from now on carries its captions in fifteen languages. English, Arabic, Farsi, Hebrew, Urdu, French, German, Spanish, Italian, Russian, Indonesian, Swahili, Chinese, Japanese and Greek. Where a video has more than one, a language menu appears beside the Captions button.
• The list is deliberately not just “the big ones”. Swahili and Indonesian are there because of where the people reading this actually are. It is a starting point rather than a boundary — there is a link on the watching videos page to tell us which language we have missed, and someone asking is the most useful signal we have about which to add next.
• The spoken audio stays in English. What changes is the text on screen. If you are reading along in another language you will see the English printed into the picture at the bottom and your language at the top — which turns out to be the better arrangement for anyone learning, rather than a compromise.
• Choosing a language costs you nothing to load. Only the one you have picked is ever downloaded, so a video offering fifteen languages is exactly as light as one offering a single language. If you never turn captions on, none of them is fetched at all. We checked this rather than assuming it.
• These are machine translations and we would rather say so. Nobody has read them. For ordinary description they are good; for anything turning on a precise word — a line of scripture, a name with a settled form in your language — the English is the text to rely on, and a translated line marked with an asterisk is telling you exactly that. We took the same view as we did with pronunciation elsewhere: an unreviewed rendering warns you, it never blocks you. Hiding a language until someone has checked it would mean most people get nothing, for a long time, in the name of accuracy.
• The hard part was making a bad translation impossible to publish quietly. Subtitles are a list of lines, each pinned to a moment in the audio. If the translation comes back with two lines merged into one, everything after that point is attached to the wrong moment — and the file is still perfectly valid, so nothing complains. We now count the lines back and refuse a language outright rather than store one that looks fine and is a second adrift. A language that fails simply does not appear, and is picked up automatically later.
• That happened on the very first run, which is the useful part. Fourteen languages, thirteen came back clean, one came back malformed. It was refused, the video was unaffected, and it succeeded when retried a moment later. We would rather find that on day one than discover it months from now in a language none of us reads.
• Only videos made after August 2026 can have captions at all, in any language, and that is a real limitation rather than a rollout. The timings that record when each word is spoken are worked out while a video is being made, and older videos did not keep them. There is nothing left to translate from.
• Nothing to do at your end.

A tidy-up that reported success every time it ran

• When a video was remade, the old one was cleared away and two of its companion files were not. Each video is stored alongside a thumbnail and a small description file. Remaking a video replaced all three, but the tidy-up that removes the previous set was looking for those two under names they had never been saved with.
• It reported a clean tidy-up every single time. Asking storage to delete something that is not there is not an error — it is treated as “nothing to do”, and succeeds. So the log said two files had been removed, truthfully describing what the code had asked for and not at all what happened. This is our least favourite kind of bug: the result looks identical whether it worked or not, so nothing ever points at it.
• Fixed at the naming end rather than the deleting end, so the three files that belong to one video are now saved under matching names and the tidy-up finds them. We also made the delete able to say “there was nothing there”, and the logs now report what was actually removed rather than what was attempted.
• Then we counted, because a fix with no measurement looks the same as no fix. The left-behind files were found and cleared, and a second pass confirmed none remained. The pattern was unmistakable once counted — the leftovers came in exact pairs, one thumbnail and one description file per remake, which is precisely what the fault predicts.
• Nothing to do at your end.