Changelog — 2.55–2.63

Releases 2.55.0 through 2.63.0. Current releases are on the main changelog, and every band is listed in the release archive.

Loading audio...

Version 2.63.0

August 4, 2026

Our release process would wait forty minutes to tell us about a problem it could have spotted in one

  • Putting a new version live involves waiting for the software to be built and packaged, and we allow up to forty minutes for it. That is a sensible amount of time for a real build. The trouble is that a build which never started looks exactly like a build that is taking a while — so the wait ran its full course and then reported something that read like “the machines are busy”, when the truth was “nothing was ever asked to happen”.
  • It now asks that question directly, up front, and it takes about a minute. We had already built the tool that can answer it; this connects it to the release process, so the cheapest check runs first and the expensive wait only happens when there is genuinely something to wait for.
  • It is deliberately allowed to say “I do not know”, and that answer never stops a release. If the new check cannot reach the system it needs to ask, or is confused for any reason, the release simply carries on to the original forty-minute wait. Only a clear, confirmed “nothing was triggered” stops anything. A helpful extra check that becomes a new way for releases to fail is worse than not having it.
  • A release that got interrupted part-way could not be picked up again. This happened to us for real: a version was prepared and its software built, then the process stopped one step before going live, and there was no way to continue — the safety checks that stop you releasing the same version twice also stopped us finishing the one we had started. We escaped it by skipping to the next version number, which is not something anyone should be taught to do.
  • There is now a proper way to continue, and it refuses unless it can prove where it got to. It checks that the version was really prepared, that it matches the code we are actually looking at, and that the software was really built — and if any of those is not true it says which one, because “this was interrupted” and “someone has changed things since” need completely different responses.
  • Resuming cannot skip the safety check, and not because we remembered to include it. Before anything goes live we deliberately confirm our verification actually notices when something is wrong — a check that always says “fine” is worse than no check. Rather than trusting ourselves to repeat that in the resume path, we arranged things so there is only one route to going live, and it passes through that confirmation.
  • We also corrected our own written instructions, which had gone out of date a single day after being checked. A note recorded on the 2nd of August was made wrong by work finished on the 3rd, and it sat on the page people read when a release misbehaves. It is fixed, and the surrounding advice now tells you to re-run the command rather than believe the paragraph.
  • Nothing to do at your end. This is entirely about how we put new versions live.

Version 2.62.0

August 4, 2026

We could not answer “did that change actually rebuild anything?” without opening a web page and squinting at it

  • The fix from earlier today now applies everywhere, so editing documentation no longer rebuilds and re-publishes software that has not changed. We proved it worked on one service first, then applied the same change to the other seven — and we worked out each one’s settings by reading them rather than typing them, because three of them are named after things that no longer exist.
  • Checking it worked used to mean opening a page and reading a list by eye. That does not scale to eight, and it leaves no record anyone can repeat. There is now a command that answers it directly, and it can be told what it expects to see and what it expects not to see.
  • The obvious way to ask the question gave the wrong answer, in the dangerous direction. There are two similar-looking ways to query our build system: one lists work that has been picked up, the other lists work that has been created. On a busy day the first reports a job as simply not existing when it is really sitting in a queue — which reads as “the change broke things” when the change was working perfectly.
  • It also refuses to answer when it does not yet know. “This did not happen” and “this has not happened yet” look identical for the first minute or two, and the tool now waits for proof the change has been seen at all before it will make any claim. Saying “I do not know” is genuinely more useful there than a confident wrong answer.
  • Nothing to do at your end. This is entirely about our own build machinery.

Version 2.61.3

August 4, 2026

Correcting a typo in our own documentation was rebuilding and re-publishing software that had not changed

  • Editing a few explanatory files was enough to trigger a full rebuild of several separate pieces of our software. The rebuilt versions were identical, so nothing was ever wrong with what came out — but it used a lot of machine time we did not need to spend, on machines we would rather were doing something useful.
  • It also quietly destroyed a piece of information we rely on. We use “when was this last rebuilt?” to answer “how old is the data inside it?” — and rebuilding for no reason resets that clock. One of our services uses exactly that to tell us how stale its reference data is.
  • The fix is a one-line instruction to ignore documentation changes — and we have applied it to exactly one service on purpose. The software that runs our builds understands only part of the instruction language it borrows from, and an instruction it does not recognise could quietly mean “ignore everything” rather than “ignore documentation”. That would leave a service with no automatic builds at all, and we would not find out until we needed one.
  • So we are testing it in both directions before applying it anywhere else. A documentation change must stop triggering a rebuild, and — the part that actually matters — a real code change must still trigger one. Confirming only the first would not tell us whether the instruction was working or had simply switched everything off.
  • Nothing to do at your end. This is entirely about our own build machinery.

Version 2.61.2

August 4, 2026

Our undo button was working out what to undo by asking the wrong thing

  • It was reading our own records rather than asking the live site what it was running. Those two normally say the same thing, which is why it looked right every time we tried it. They stopped agreeing the moment we prepared a version and then did not put it live — and at that point the undo was describing a step backwards from something that had never been live in the first place.
  • Following that plan would have appeared to work perfectly and done nothing at all. Every command would have reported success, everything would have looked healthy, and the site would have been completely unchanged. That is the worst possible shape for a mistake, because it would happen at the exact moment something was already wrong and nobody had attention to spare.
  • It now asks the live site directly, and refuses to guess. If it cannot reach the site it stops and says it does not know, rather than falling back to the assumption that caused this. And if the version you asked to go back to is the one already running, it refuses outright instead of handing you a page of instructions that would achieve nothing.
  • It also now tells you whether the version you are going back to has ever actually run. A version being prepared is not the same as a version having been live. Going backwards to something nobody has ever seen working is not a step towards safety, and during a problem that distinction matters more than usual.
  • We found this by reading the output carefully, not by anything failing. Nothing was broken and no check complained — the first line of a plan was simply wrong, and we were going slowly enough to notice. We have written up how we found it alongside the fix.
  • Nothing to do at your end. This is entirely about our own tools.

Version 2.61.1

August 4, 2026

The first real use of our new release process stopped itself one step before going live, for a reason we had not thought of

  • It halted saying we had asked it to stop, and we had not. While waiting for the new version to be built, the tool checks the server repeatedly. Each of those checks was quietly consuming the answers we had already given it for the later steps. By the time it needed one, there was nothing left to read.
  • Nothing reached the live site. It stopped one step before anything would have changed, confirmed the site was untouched, and printed the exact command to undo the part it had done. We checked afterwards: not a single byte of the live site had changed.
  • The reason it stopped safely, rather than carrying on, was a small decision made much earlier. Every question it asks defaults to “no” rather than “yes”. So when it ran out of answers, it treated silence as “stop”. Had that default been the other way round, it would have gone ahead unasked. We did not predict this failure — the habit caught it, which is rather the point of having habits.
  • We have added a check so this exact mistake cannot come back. It also had to be taught the difference between a command the tool runs itself and one it merely prints for a person to read, which are not the same thing.
  • One thing is deliberately still missing. If a release is interrupted partway, the tool cannot yet pick up where it left off. We chose not to add that in a hurry on the same day, because rushing an untested recovery path onto an untested tool is how the next problem gets built. It is written down and will be done properly.
  • Nothing to do at your end. This is entirely about how we put the site live.

Version 2.61.0

August 4, 2026

Putting a new version of the site live was fourteen steps done by hand, and some of them failed silently

  • Releasing is now a single guided script that explains each step and asks before it does anything. It checks the work, publishes the release, waits for the new version to be built, puts it live, and then proves the site is actually serving it. Someone who is not a specialist can follow it, which was the point — the old process lived across five documents and depended on remembering the order.
  • The step we most needed it for was the one that failed quietly. If you skipped a single line, the site would carry on serving the old version while every command reported success and every indicator stayed green. Nothing in the process could see it. The script now proves the new version is really live before it says it is done, and it refuses to continue if it cannot.
  • We also built the undo — and then deliberately stopped short of letting it press the button. It works out exactly what undoing a release would involve and prints the instructions; it does not run them. We can prove the thinking is right without anything running; we cannot prove the doing is right until we have rehearsed it somewhere safe, and we do not yet have anywhere safe. So it says, in as many words, that the plan looks sound but is not tested. A plan that quietly implied otherwise would be worse than no plan.
  • The part that decides whether an undo is even safe is deliberately pessimistic. Going back to an older version does not undo changes to how information is stored, so the older version has to be able to cope with the newer arrangement. Rather than listing the changes we know are dangerous, it lists the ones we can prove are harmless and stops at anything else. That will occasionally be over-cautious. It cannot be quietly wrong, which is the trade we wanted.
  • Building it found three faults in itself, and all three were only visible by running it. The code read perfectly well in each case. One would have sent someone to investigate a site that was working fine; another told a machine to run a check that does not exist there. We have written down what each one was rather than quietly correcting them.
  • It deliberately knows nothing of its own. Every fact it uses is read from wherever that fact actually lives, and where it cannot read something it refuses instead of assuming. We found that three of our nine build configurations are named after things that no longer exist, so anything written from memory would have been wrong about a third of them.
  • Nothing to do at your end. This is entirely about how we put the site live; the site itself is unchanged.

Version 2.60.0

August 3, 2026

You can now see which version of the site you are looking at, at the bottom of every page

  • The version number now appears in the footer, and it links here. If something looks different from how you remember it, that number tells you which release you are on and what changed in it. It also means we can tell at a glance what is actually live, which previously took running a diagnostic.
  • Adding it created a small risk, and we closed it in the same change. A version number written by hand in one more place is a version number that will eventually be wrong — it has happened here before, sitting seven releases out of date on the page most people open first. The build now refuses to complete if the footer disagrees with anywhere else the version is recorded.
  • We also stopped writing down how many places it lives. That number was recorded in fifteen different notes, and adding one more home made every one of them wrong at once — silently, because they were just sentences. The check now works it out and reports it, rather than trusting what someone typed.
  • And we brought our release instructions in line with what a release actually does now. They were written before a recent change and quietly described a process that no longer matched — a single release now rebuilds fourteen separate pieces of software rather than one. We rewrote them against a real release rather than from memory.
  • Nothing to do at your end. The footer is the only visible change.

Version 2.59.0

August 3, 2026

Our own build system had sixteen lines of configuration that had never once done anything

  • Eight of the nine pieces of software we build automatically were never labelled with a version number. The instruction to label them was there, in every file, and could never take effect — it only applies at the moment a release is named, and those eight were never told to pay attention at that moment.
  • What that cost us was the ability to go backwards cleanly. Undoing a bad update meant looking up an anonymous identifier rather than asking for the previous version by name. Now each piece carries its version.
  • The obvious fix would have broken all eight at once. Switching the instruction on, without a second change alongside it, produces an invalid label and every build fails immediately. We know because it happened once before, to a different part of the system. Both changes went in together.
  • And it turned out to depend on something written down nowhere. The fix only works on a recent enough version of the software that runs our builds; on an older one it would have quietly done nothing at all while appearing to succeed. We checked the actual version rather than assuming, and wrote down both the answer and how to re-check it.
  • This was the same mistake copied eight times. One file had the problem, someone fixed it, and the eight copies made from it kept the fault. The check we have added now catches a ninth copy automatically, which is the only reliable way to stop that pattern.
  • We were careful about what we can honestly claim. The new check proves the setup is capable of working; only an actual release proves it does. That distinction is written down rather than glossed over, and the next release is the real test.
  • Testing this change found a second, unrelated fault — builds were failing after they had already succeeded. Each build saves a copy of its work to make the next one faster. When saving that copy failed, the whole build was marked as failed, even though the software it produced was complete and correct. Saving the copy is now allowed to fail quietly, because it is a convenience and the software is the thing that matters.
  • We were wrong about the cause twice before reading the evidence properly. The first two explanations were confident, reasonable, and both wrong — we had assumed something was being downloaded too slowly, when in fact something was being uploaded. The actual log settled it in seconds. We have written down what we got wrong alongside what we fixed.
  • Nothing to do at your end. This is entirely about how we build and release the software — the site itself is unchanged.

Version 2.58.0

August 3, 2026

When a long job was interrupted we told everyone their work was gone, and usually it was not

  • Interrupted work used to report the same thing regardless of what had actually happened. If the server restarted mid-way through generating a video or a set of illustrations, the entry said “Nothing was saved — start it again.” For most kinds of work that was simply untrue: the finished pieces had already been stored safely.
  • The cost of that was real, not cosmetic. Someone reading it would start a half-hour job again from scratch, on top of one that had already finished. The message now says which of the two actually happened.
  • Each kind of job now carries its own record of whether its output is saved as it goes. That was previously written down only as a note for developers to read, which meant nothing could act on it. Adding a new kind of job without saying which it is now stops the build.
  • We were deliberately cautious about which way to be wrong. Telling someone their work is gone when it is not costs them half an hour. The opposite costs them refreshing a page. Anything we were unsure about takes the cautious wording.
  • Some interrupted jobs could never be tidied up at all. A job only got cleaned up if it had been checking in regularly, and not all of them do — so some had been sitting marked “in progress” since March. There is now a time limit for every kind of job, set generously so it can only catch something genuinely stuck.
  • We resisted the obvious fix, on purpose. The clean-up routine has never actually run. Switching it on would have been a one-line change — and would have made last week’s incident worse, marking a video that was still being made as failed. It gets switched on once the rest is trustworthy.
  • And we added the missing clue. The reason last week took a day to understand is that a job stopped reporting in and left no explanation. It now says when it was prevented from checking in, and for how long.
  • Nothing to do at your end. If you have seen “Nothing was saved” on something that was interrupted, it is worth reloading before re-running it — there is a fair chance the work is already there.

Version 2.57.0

August 3, 2026

One slow job could quietly stop the whole site responding, and we had been reading the symptom wrong

  • The site runs as a single program, and certain audio tasks were stopping it dead. While one of them prepared a file, nothing else could happen — no other page could be served, no background timer could tick. It was invisible from outside: the page you asked for still arrived, just later, and every health indicator stayed green throughout.
  • Twelve of the seventeen places this happened are fixed. They now run alongside everything else instead of in front of it. The remaining five sit in a file that is being rewritten shortly for another reason, and doing them now would mean doing them twice.
  • We had been blaming the wrong thing for a week. A long job appeared to die after four minutes. It had not — it was working the whole time, but the clock that reports “still alive” had been frozen along with everything else. The report was wrong, not the job.
  • Measuring it first changed what we fixed. Our own description of the problem turned out to be wrong in three places, and the biggest culprit was not the one we had written down. Fixing what the note said would have left the real one untouched.
  • One part stopped needing the slow step at all. Building spoken pages, we were running a separate program to produce each pause between paragraphs — silence, generated the expensive way, once per gap. It is now simply calculated, so it costs nothing.
  • We can now prove the fix rather than assert it. There is a check that deliberately reproduces the old fault alongside the new behaviour and compares them. A test that has never been seen to fail proves nothing, so this one has to demonstrate it can, every time it runs.
  • And the rule is enforced, not just written down. Anything reintroducing the old pattern now fails automatically. Written-down rules here have been quietly broken four times before; the ones the machine checks have not.
  • We also found a rule that had been enforced for weeks but never documented. The page explaining our rules was missing one of them, which is the same failure it warns about elsewhere.
  • Nothing to do at your end. Audio and video features behave exactly as before — the site simply stays responsive while they work.

Version 2.56.0

August 3, 2026

These release notes were getting long again, so we split them before it became a problem rather than after

  • This page had grown to the point where it was worth dividing again. We measured rather than guessed: roughly three more releases before it would have become slow to load. Long enough to ignore, short enough to regret ignoring.
  • Releases 2.49 to 2.54 have moved to a page of their own. This page keeps the newest ones, and every older page is listed in the release archive, linked at the foot.
  • Any link you have saved still works. A link to an older release is forwarded automatically to whichever page now holds it. That has been true at every split, and we test it rather than assume it.
  • We did it now on purpose. The next piece of work is a two-part change, and left alone this split would have landed in the middle of it — which is exactly when a fiddly job gets done badly.
  • The obvious place to divide was the wrong one, and we nearly took it. Splitting after the newest release would have left this page empty, with nothing marked as current. We cut one release earlier and wrote down why, so the next person does not have to discover it the hard way.
  • Three of our own notes turned out to contain counts that this change made wrong. Each sat within a few lines of a warning not to write counts down. We replaced them with instructions for taking the measurement instead — a number written beside the thing it counts will always drift away from it.
  • We deliberately broke the safeguards to see which one would notice, and were wrong about the answer. We expected one check to catch a bad split; it did not, and a different one did. Both are now known to be necessary, and the reasoning is recorded rather than left as folklore.
  • Nothing to do at your end. Nothing about the site changed — only how these notes are organised.

Version 2.55.1

August 2, 2026

A test that checked our transcription was itself relying on the thing we are about to retire

  • We have a check that proves our transcription and caption-timing actually work. It made a short piece of speech, transcribed it, and timed the words — a genuine end-to-end proof rather than just asking each part whether it felt well.
  • But it made that speech using an older voice system we are retiring. Removing that system would have quietly removed the check’s own starting material. So we fixed the check first, before touching anything else.
  • It now uses one fixed, stored recording — and that made the check stronger, not weaker. The old voice produced something slightly different every time, so the only thing we could safely ask was “did you return anything?” With a recording that never changes, we can now ask whether the transcription is actually correct. It is, word for word.
  • We deliberately broke it twice to prove it still notices. Once by removing the recording, once by replacing it with silence. Both were caught. A test that has only ever been seen passing has not been shown to work.
  • That second attempt found a real weakness we had not suspected, and we fixed it. Given complete silence, the word-timing step confidently reported timings anyway — it is told in advance what the words should be, so it produces plausible answers for audio containing nothing. It also reports how confident it is, and on silence that figure was around two thousand times worse than on real speech. We now check that figure, so silence is caught twice over instead of slipping past.
  • While measuring it we found a second problem waiting to happen. If that confidence figure ever went missing, the code would have quietly substituted a high value — turning “we have no idea” into “we are certain”, and defeating the very check we had just added. The check now rejects that value as impossible, so it cannot be fooled that way.
  • Nothing to do at your end. No change to the site. This is groundwork so that retiring the old voice system next cannot quietly blind the checks that would tell us if something broke.

Version 2.55.0

August 2, 2026

When a page repeated a heading, its contents links all led to the same spot

  • Longer pieces have a contents list you can jump from. Each entry needs its own address on the page. When a piece used the same heading twice — and one of our talks repeats “Voices of the Scholars” five times — every one of those entries was given the same address, so they all jumped to the first.
  • Repeated headings now get their own addresses, and the first one keeps the address it always had. That last part matters: any link you have already saved or shared still lands exactly where it did.
  • This had to be fixed before our reading room opens. That reading room will remember where you got to in a long talk, and it remembers it by heading. If two headings share an address, it would keep returning you to the wrong one — permanently, and with nothing to suggest anything was amiss.
  • The explanation we had written down was wrong, and we checked rather than assuming. Our own notes said the problem was Arabic headings collapsing together. Measured across everything we hold: not one heading is Arabic-only. The material is full of Arabic in the body, but the headings are almost entirely English — and repeated English headings were the whole problem.
  • The tests are written to fail if the fix is ever removed, and we proved that before trusting them. We deliberately broke it two different ways first and watched the right tests go red each time. A test that has only ever been seen passing has not been shown to work.
  • Nothing to do at your end. Any contents link you have saved still works. Repeated headings simply now go where they say they do.