Changelog — 2.74–2.83

Releases 2.74.0 through 2.83.0. Current releases are on the main changelog, and every band is listed in the release archive.

Loading audio...

Version 2.83.0

August 6, 2026

We found that most of our largest commentary was never actually being read, and had not been for a long time

  • To find a passage by meaning rather than by keyword, we first convert it into a form a computer can compare. We had assumed that step reads whatever we hand it. It does not. It reads the opening and quietly discards the rest — no error, no warning, and a result that looks exactly like a correct one.
  • We measured where it stops instead of trusting the documentation, and the answer was worse in Arabic than in English. It reads roughly the first two thousand characters of English, but only around eight hundred of Arabic. Nothing we were given says that, and it matters enormously to us: much of what we hold is Arabic, and it was the half being cut shortest.
  • The consequence for our largest commentary was total. It was stored one whole section at a time, and those sections are long — so for the overwhelming majority of them, almost none of the text ever reached that step. Worse, the part that did reach it was the opening, and every section opens the same way. So they all came out looking identical to each other and unlike any question anyone would ask. That commentary was returning nothing at all, for anything, and had been for months.
  • It is now stored at the size of an idea rather than the size of a chapter. Those finer divisions already existed in the source and had simply never been used. Anything still too long is split at a sentence or paragraph break rather than mid-thought — and the splitter understands Arabic and Persian punctuation, because one that only knows English full stops would fail worst on exactly the material with the least room.
  • The Quran was searchable only in its Arabic, although we already held the English. The translation was sitting in the same place, unused, so a question asked in English was being matched against Arabic alone. Each language is now stored in its own right, so a question finds the text in the language it was asked in. We were careful to use the same translation a reader actually sees on the page — a reference that leads somewhere different is worse than none, because it looks correct.
  • The rule about what our writing may quote is now applied when searching, not only when storing. Last week we made the store itself refuse a wrongly-labelled passage. The search had no such rule at all until now.
  • And we planted markers in the real library that must never be quoted. A rule that only ever sees material it should allow will pass whether or not it works. So there is now a clearly-labelled marker for each excluded tradition, written to sit near ordinary questions rather than in a forgotten corner — and we check both that it stays out, and that it can be reached when deliberately allowed in. Without that second check, “it did not appear” would be satisfied by it simply not being there.
  • Our own tooling told us a job had succeeded when it had done nothing at all. A run failed every single one of twelve thousand records and still reported success, because it only reported *finishing*. That is the exact fault this whole piece of work exists to remove, sitting in the tool built to report it. It now refuses to claim success when it wrote nothing.
  • Nothing has changed on the site yet, and no published writing has been altered. The next step is to re-run the blind comparison that tells us whether any of this actually improved what gets quoted. We will say either way.
  • Nothing to do at your end.

Version 2.82.1

August 6, 2026

Our automated checks reported a failure that wasn’t one, and the reason they did is worth removing

  • Every change we make is checked automatically before it goes anywhere. One of those checks reported a failure — and it had stopped before running a single test. It fell over while preparing the machine that would have run them.
  • Two checks were running at the same moment and both tried to update the same shared tool. Updating it involves moving it aside briefly, and two things moving the same thing at once is a collision. One of them lost. It only happened because the two ran unusually close together that afternoon.
  • The change itself was fine, and we could show it rather than assume it. The earlier check on identical content had already passed, and we confirmed the two were byte-for-byte the same rather than taking it on trust. Re-running the failed check — nothing altered, just no longer overlapping — passed. That is what tells you the fault was the collision and not the work.
  • We fixed it anyway, and the reason is the important part. A check that fails now and then, and passes if you press the button again, is on its way to being ignored — and this one runs immediately before anything is released. A warning people learn to click past is worse than no warning, because it still looks like it is watching.
  • The fix is to stop doing the risky step when it isn’t needed. The tool is now checked before being replaced, so almost every run skips the operation that can collide. If it does collide, it waits and looks again — and if it genuinely cannot be prepared, it stops there and says so plainly, rather than continuing and failing later with a confusing explanation.
  • Nothing to do at your end. No published content was involved at any point.

Version 2.82.0

August 6, 2026

The rule about what our writing may quote from is now enforced where the material is stored, not only where it is searched

  • Every source text we hold is labelled with which tradition it belongs to and what kind of writing it is — revealed text, reported sayings, commentary, doctrine, history, and so on. Our published writing may only quote from a specific combination of the two.
  • Until now that rule was applied when searching. It is now also applied when storing. The store itself will refuse a label that is not one of the agreed ones, and refuse an empty passage outright.
  • That closes a failure that would have been invisible. A label with a stray capital letter or a trailing space looks correct to a human and matches nothing at all — so the passage would simply never be quoted, forever, with no error anywhere. The only symptom would be a body of material that quietly underperforms for no apparent reason. Now it is refused at the point of writing, and says which value was wrong.
  • Anything unlabelled defaults to “unknown”, and unknown material is never quoted. If we get a classification wrong, a source quietly does not appear — noticeable and fixable. The alternative failure, where something is quoted because the default was generous, is the one you cannot take back.
  • And we made the rule testable by planting something it must refuse. A filter that only ever sees material it should allow will pass whether or not it works — there is nothing there for it to wrongly let through. So the tests plant a marker from an excluded tradition, sitting right alongside a permitted passage and equally likely to be picked, and check it is kept out. Then they check the same marker IS reachable when its own tradition is allowed, which proves it was really there to be refused.
  • Then we broke the rule eight different ways to confirm the tests notice. Each time, the checks that failed were exactly the ones describing that fault. That is the part worth having: not that the tests pass, but that we have watched them fail for the right reason.
  • Three of our own tests failed against perfectly correct code, and each was worth it. One assumed two lists of numbers were unalike when they were in fact pointing the same way; another depended on the exact spacing of a generated file. Both were the test being wrong, and finding out that way is much cheaper than trusting them later.
  • Nothing is visible on the site yet and no published writing has changed. This is the foundation the rebuilt library will be stored in. Next it gets filled, and then we re-run the comparison that tells us whether any of it helped.
  • Nothing to do at your end.

Version 2.81.0

August 6, 2026

Before trusting a measurement, we check the instrument can report a bad result — and this one had never been checked

  • We are rebuilding how the site finds relevant passages, and the whole rebuild is judged by one comparison. A person is shown a story idea and two passages, told nothing about where either came from, and asked which suits it better. That single number decides whether the new approach is an improvement or a step backwards.
  • The tool that builds those comparison sheets had never been tested. It worked, and it had produced a result we were already quoting. What it lacked was any demonstration that it would notice if it were wrong — and everything about it that could be wrong produces a sheet that looks completely normal.
  • The worst case would have reversed the answer without changing anything you could see. The sheet records which side holds which passage, and separately decides what a click means. If those two ever disagreed, every result would come out backwards — the losing approach would be reported as the winner, and the page would look identical either way.
  • So the tool now tests itself, and we broke it six different ways to prove the tests notice. Each time, the checks that failed were exactly the ones describing that fault. That is the part worth having: not that the tests pass, but that we have watched them fail for the right reason.
  • Two of the checks follow the whole path at once. They simulate someone choosing the passage that is genuinely in front of them, and confirm the result is recorded against the right approach — four hundred times over. A mistake anywhere along that path fails them, whichever end it was introduced at.
  • We also found the tool was reaching into live data simply by being opened. Anyone examining how it worked would, without being asked, run hundreds of live queries and overwrite the existing sheet. A tool that does its work the moment you look at it cannot be inspected without setting it off. Fixed.
  • Why this is worth a release of its own. The next step rebuilds the material behind Reflections and Vignettes and then re-runs that comparison to see whether it helped. A comparison made with an instrument nobody has seen produce a bad reading is not a comparison — it is a number that will agree with whatever it is given.
  • Nothing to do at your end. No published writing has changed.

Version 2.80.0

August 6, 2026

Our source library labelled every book with the tradition of the library itself — and about one book in six was from a different school entirely

  • A large collection of classical works was being prepared for use behind Reflections and Vignettes. Every book in it carried the same label saying which tradition it belonged to. That label was applied once, to the collection as a whole, and it describes whose library it is — not what any individual book is. A seminary library naturally holds the other schools’ sources, because students study them.
  • Roughly one book in six was from another school, and every one of them said otherwise. Had we trusted the label, those books would have sat inside the set our published writing is allowed to quote from, correctly stamped and completely wrong. Nothing would have failed or looked unusual. We would have found out from a reader.
  • The fix is a smaller job than it sounds, and that is the point. The collection sorts itself into a few dozen clearly-named subject headings, and each heading answers the question unambiguously. Deciding those once settles almost every book at a stroke — and a few dozen decisions can be read and checked by a person, which thousands cannot.
  • Anything we are not sure about is marked unknown, and unknown material is never quoted. This is deliberate and it is the whole safety of the arrangement. If we get a classification wrong, the result is that a source quietly does not appear — noticeable, harmless, and fixable. The alternative failure, where an unclassified work is quoted because the default was generous, is the one you cannot take back.
  • We also record what KIND of text something is, not just which tradition — and that turns out to matter more than we expected. A speech given last year by a contemporary political figure genuinely belongs to our tradition. It is not the same thing as the words of the Imams, and it should not be quoted as though it were. Sorting by tradition alone would have made them equivalent. Sorting by kind as well is what keeps them apart.
  • Having measured the collection properly, we deliberately widened what may be quoted. Works of doctrine and of biography are now included, where before only revealed text, reported sayings and commentary were. That was a judgement made after seeing what the collection actually contains, and we have written down that it was a decision rather than an accident — so nobody later mistakes it for something that drifted.
  • We tested the boundary by trying to break it rather than by confirming it works. A test that checks the right things are allowed through will happily pass even if everything is allowed through. So the checks that carry the weight are the ones naming material that must be refused — and we verified them by deliberately mis-labelling a book from another school and confirming the checks caught it.
  • One more correction worth admitting. We had recorded the collection as containing several thousand separate books. It contains a few hundred works, split across several thousand files — one of them alone accounting for about a tenth of everything. That matters: a collection of thousands of distinct books is far more varied than a few hundred, and search results need to be spread across genuinely different sources rather than across many pieces of the same one.
  • None of this is visible on the site yet, and no published writing has changed. This is preparation. We have also opened a piece of work to explain all of it properly in a help page, alongside the one we are writing on how we use AI — because if we are going to quote from a body of source material, you are entitled to know how we decided what is in it.
  • Nothing to do at your end.

Version 2.79.0

August 6, 2026

We have a rule that a test must be watched failing before it is trusted — so we pointed it at our own tests, and three of the four could not fail at all

  • Yesterday’s release added a new piece of the system, and with it a set of checks that run after every update. Those checks come with a companion whose only job is to prove they still work — by deliberately breaking each one and confirming it notices. It is the difference between “everything passed” and “everything passed, and we know a failure would have shown.”
  • This time we tested the prover itself, and it was mostly theatre. Three of its four proofs were comparing two fixed values written into the file — they printed a pass no matter what the real check did. We confirmed it the honest way: we broke each real check on purpose, and the prover cheerfully reported all clear.
  • The worst of the three guarded the single thing that piece of the system exists to do. It would have reported success while the check protecting it was reversed.
  • The fix generalises, which is why it is worth writing down. A proof only covers what it genuinely shares with the thing it is proving. Reciting the same comparison a second time proves only that you can recite it. Each check is now driven through the very same code the real one uses, and each is tested in both directions — that it fails when it should, and still passes when it should. A check that can never succeed is as useless as one that can never fail, and only testing both catches it.
  • Then the same fault turned up in the new tool we wrote that morning, found by the same method and fixed the same day. It is a more encouraging result than it sounds: the technique works on the work of the person using it, not just on inherited code.
  • We also replaced a speed figure we had been relying on that could not be checked. It existed only as a sentence, with no way to take the measurement again — and in a unit that quietly means very different things depending on the length of the text involved. It is now something anyone can re-run, and every number it reports carries the context needed to read it honestly.
  • The practical effect is on the library behind Reflections and Vignettes. Some of that material has not been reachable by search at all. Repairing it was previously slow enough that we kept putting it off; that constraint is now gone, so it can be done properly rather than sparingly, and redone whenever the material changes.
  • The same day, checking a related assumption turned up something larger. The scripture we quote most often had been indexed in its original Arabic only — even though an English translation was sitting unused in the very same source file. So the collection our writing draws on most was the one we had given the search the least to work with, in the language people actually ask questions in.
  • And a belief we had written down about how well searching across languages works turned out to have measured the wrong thing. It recorded how similar a passage looked, not whether it actually came out ahead of unrelated ones. Measured properly, a passage in Arabic loses a small but consistent amount of ground to its own English translation — small enough to ignore, until you notice it is about the same size as the entire signal telling relevant from irrelevant.
  • All of it is now written down rather than living in one conversation, with the command to re-take every measurement beside it. A figure with no way to check it again is how we ended up trusting three of the things above.
  • Nothing to do at your end, and nothing has changed in what you read. This is groundwork, and an honest account of several things we had been trusting without cause.

Version 2.78.0

August 6, 2026

We went looking for why our writing sometimes cites the wrong passage, and found four separate reasons — two of them our own measurements being wrong

  • When we write a reflection or a vignette, we first search our library for a relevant passage. We have been improving that search, and this release is mostly about what we learned while measuring it — including where we had measured it badly.
  • We asked a person to judge, blind, whether the passage our search found suited the story better than the one we pick today. Twenty-five real examples, with no indication of which was which. The answer was not the one we expected: in four cases out of ten neither passage was any good, and where a preference was expressed it was too close to call.
  • That killed a piece of work we were about to start, which is the best possible outcome for twenty minutes of someone’s time. We had been planning to replace how we choose passages, on the assumption the new way was better. It is not better; it is different. So we stopped.
  • The real problem turned out to be that we were asking the wrong question. A story premise describes a plot — a girl finds an injured bird. The passage that suits it is about the lesson: compassion, responsibility, letting go. We were searching with the plot, and getting back passages about birds. We now work out the theme first and search for that as well.
  • Then a much odder thing surfaced. We tried searching for deliberate nonsense — random letters, no words — and it came back with passages scored as a better match than a genuine question about religious dues had managed. Very short verses, a few words long, turn out to sit close to everything, so they surface no matter what you ask.
  • We had also mismeasured our own library twice, and corrected both. One collection appeared to be swamping the results; measured properly against real examples rather than the test questions we had invented, it was a fifth of what we reported. Test questions we choose ourselves flatter whatever we are hoping to find.
  • And one entire commentary has been unreachable this whole time. It is stored in very large pieces, and we were only ever looking at the opening of each — which is the same few lines of praise every time. Buried inside it are roughly twelve thousand individually quotable narrations that nothing could find.
  • We also built a machine to do this searching properly. The part that reads and compares text has been running on general-purpose hardware because of a technical constraint we could not work around in place. It now has its own machine, which makes this kind of work hours rather than days.
  • Nothing to do at your end, and nothing has changed in what you read yet. This release is groundwork and honest accounting.

Version 2.77.0

August 5, 2026

Half our library was on the shelves but invisible to search, and the collection we quote most was part of it

  • We keep six collections of verses and narrations, and search could only reach three of them. The other three — Nahjul Balagha, Uyun Akhbar al-Ridha and Ghurur al-Hikam — were sitting on our shelves and could not be found by anything. Not incomplete: absent.
  • All six can now be searched. That is roughly two and a half times as much material available to look through when we write, and it took a few hours of steady work by the machine that does the reading.
  • The one we were missing most was the one we quote most. Of the passages our illustrated pieces cite, a large share come from Ghurur al-Hikam — and it was the largest of the three absent collections. So the gap was not evenly spread; it was concentrated exactly where it would do most harm.
  • We knew it was done because a page told us, rather than because we counted. Last release added a screen that works out what we hold against what can be searched and reports the difference. It named these three. When the work finished, it stopped naming them, on its own, with nothing rewritten. That is the whole point of building it that way.
  • Our own progress check told us we had finished about two hours before we had, and we caught it by chance. It was working out whether a collection was searchable by asking whether a place existed to put it — not whether anything was in there. Since all three places get made at once, the first collection finishing made all three look done. An empty shelf is exactly as unsearchable as no shelf, and the screen whose one job is to say so was saying the opposite.
  • Nothing went wrong while it was wrong, which is the part worth dwelling on. No warning, no error, no failing test — just a reassuring page. It now asks the question it was always meant to ask, and the tests for it are deliberately written around the empty case, because a test that only checks the happy ending would have passed on the broken version too.
  • Along the way we found we had been mislabelling our own sources to the AI. One commentary was being handed over under its internal filing name rather than its title, and Quranic references were carrying a stray fragment of punctuation where a chapter number should have been. Both had been wrong for a long time, and neither could ever have caused an error — the wording just quietly came out wrong.
  • That is exactly the kind of fault that survives, so we wrote a check that compares the two halves against each other. The names live in one place and the sources they describe live in another, in a different language, in a different file. Nothing had ever compared them. Now something does, and a new collection added without a proper title will stop the build rather than reach a reader.
  • We broke it on purpose twice before believing it. We put each of the two original faults back, confirmed the new check caught each one, and only then put the fix back. A test that has only ever been seen passing has not been shown to work.
  • We also cleared out addresses for equipment that no longer exists. Two retired machines were still named as fallbacks in our health checks, so those checks were reporting on nothing. And our note of where the search groundwork happens now says so plainly, after it took two people a while to remember.
  • Nothing to do at your end. You should simply find that what we write draws on more of what we hold.

Version 2.76.0

August 5, 2026

We could not tell, from any screen, whether the search behind our writing was actually working

  • When we write a reflection or a vignette, we first search our library of verses and narrations so the writing is grounded in real sources rather than invented. That search either works or it does not — and until now, finding out meant asking someone to run commands.
  • There is now a page that simply says so, by actually performing a search and showing what came back and how closely it matched. It uses the very same route the writing does, which matters more than it sounds: the library can be complete and the search still fail, and every other measure would look perfectly healthy.
  • Nothing on that page is typed in by hand. It counts what is really there each time you look. That is deliberate — a number written into a page is correct on the day it is written and quietly wrong ever after, which is a mistake we have made enough times to design against.
  • It also shows what we hold but cannot yet search. Three collections — Ghurur al-Hikam, Nahjul Balagha and Uyun Akhbar al-Ridha — are on our shelves but not yet indexed, so nothing can be found in them. That is the next piece of work, and the page worked it out rather than us remembering.
  • “Cannot be reached” and “is empty” are shown as different things, because they call for opposite responses — fix a connection, or rebuild a collection. Showing both as zero would send whoever is looking down the wrong path, confidently.
  • We tested that it can report bad news, not just good. A status page that has only ever shown green is not evidence of anything, so we forced each failure and confirmed the page says so plainly before trusting it.
  • Nothing to do at your end. This changes what we can see, not what you read.

Version 2.75.0

August 5, 2026

Most of the AI on this site was running on settings nobody had ever chosen

  • Different parts of the site ask an AI for help with different jobs — writing a reflection, suggesting a passage, describing a picture, drafting a summary. Until now we could only choose which AI does that for a handful of them.
  • Everything else quietly used a value typed into a file at some point and never revisited. Not wrong, exactly — it worked — but it was around twenty separate settings that nobody had actually decided, and no screen had ever shown. It is now one setting, on screen, that we can change deliberately.
  • Making that change do nothing was the hard part, and it is the part we are most pleased with. Switching how a decision gets made, for something used this heavily, should not alter a single outcome on the day it ships. So the new arrangement is set up to produce exactly what the old one did, and we wrote a check that compares the two for every job on the site and fails if any of them differs.
  • We then deliberately broke our own change to make sure that check works. A test that passes is only worth something if you have watched it fail for the right reason — so we swapped in the simpler design we had rejected, confirmed the check caught it, and put ours back.
  • That is how we caught something genuinely worth catching. One job — drafting an internal report when something goes wrong — is deliberately kept on our own equipment, because what it reads is our own error logs. The simpler design would have sent those to outside services. Nothing would have looked broken; it would just quietly have been happening.
  • And an older safeguard turned out to have stopped covering it. A check written precisely to protect that job still passed, because it was watching the old route while the risk had moved to a new one. Worth saying plainly: a safeguard protects the paths that existed when it was written, not the ones added afterwards.
  • The page also now says what it governs. It used to look like it controlled all of our AI settings while controlling a few of them, which is the sort of gap that makes people trust a screen more than they should.
  • Nothing to do at your end. Nothing about what you read or see has changed — deliberately, and checked.

Version 2.74.0

August 5, 2026

We had a step that tidied up transcripts automatically. It stopped running in April, and we have decided not to bring it back

  • When we turn a talk into text, there used to be a step where an AI read the transcript back and corrected its mistakes. It was built at a time when our transcription was much weaker and made errors you could spot at a glance.
  • It stopped running in April, and nothing told us. This is the same fault as the last two entries, in a third place: the rule deciding whether to run it had been written for an earlier arrangement, and when we changed which system does the work first, the rule quietly stopped matching. Four months of transcripts were produced without it and nobody noticed.
  • Our own record of it said the opposite of the truth. A line meant to note which system was doing the tidying reported that it was being handled here on our own equipment — when in fact nothing was being asked to do it at all. Anyone checking would have come away reassured.
  • Four months of nobody noticing is itself an answer, so the question stopped being “how do we fix it” and became “do we still want it”. The transcription it was built to patch up has since been replaced by something far better, so the problem it existed to solve had largely gone away.
  • The strongest reason not to rebuild it was one we did not expect. Every transcript is read by a person before anything is published. A correction written by an AI reads smoothly and confidently even when it is wrong — which makes it harder for that person to catch than an obvious mis-hearing. So it would not simply have added a little risk; it would have made the human check less effective, which is the opposite of what it was for.
  • We wrote down why, right where the step used to be. Deleting something also deletes the argument for deleting it, and the next person to find a transcription process with no tidying step would reasonably treat that as an oversight and put one back. Now they will find the reasoning instead — including what would have to go wrong for us to change our minds.
  • While we were in there, two headings in our own editing tools turned out to be genuinely hard to read. Neither had ever been told what colour to be, so both quietly borrowed our brand green — which on a dark background is close to invisible. Measured against the accessibility standard for readable text they were less than half of what they should be. This is the third time that same cause has bitten us, and each time it is a heading nobody thought to check because nothing about the code looks wrong.
  • Nothing to do at your end. Transcripts are produced exactly as they have been since April; the difference is that we now know that, and say so.