colitur.git/tools, branch v0.9.0

fix(tools): close five more ways to defeat check_citations.py

2026-08-19T13:47:37Z

Round 1 hardened check_citations.py against its own self-poisoning bug; a review defeated it again. Five fixes, in the order they were found: 1. PATTERN silenced a whole comment block, not just the entry it was attached to -- a wrong citation on a DIFFERENT, unmarked entry in the same block (e.g. [season]'s own back-to-back trailing-comment style) was never checked at all. Fixed by scoping PATTERN with the identical leading/trailing pooling rule citations already use: an entry is excluded only by its own marker, never a neighbour's. 2. Explicit per-citation ranges (introduced in round 1 to replace a blanket +-2-line tolerance) had no upper bound, reintroducing the same defect at a much larger radius (LT.txt:8600-8650 passed if the text appeared anywhere in fifty lines). Capped at MAX_RANGE_WIDTH (3 lines); anything wider is reported MALFORMED, naming the entry and the width, instead of silently accepted. 3. The "pool too thin to verify" gate counted words, not rarity -- it flagged 11 genuinely correct citations (short Latin hagionyms with only one non-stopword) CANNOT VERIFY, while a match on nothing but "classis" (507 occurrences) passed freely alongside three siblings. Replaced with a frequency table over the whole LT.txt corpus: a token's evidence is 1/(times seen), an item's evidence is its single rarest matched token (not a sum -- summing would let several merely-common words add up to "enough" between them, the same shape as the self-poisoning bug). 4. "LT.txt:12,459" (a comma typo for one number) parsed as two unrelated bare citations, 12 and 459, either of which could coincidentally match while the intended line was never checked. Detected as a thousands-separator-typo shape (a 1-2 digit token immediately followed by an exactly-3-digit one -- the only way a real LT.txt line number, which never exceeds 5 digits, splits under one comma) and rejected as malformed. 5. The self-test suite overstated its own coverage: of round 1's seven fixture cases, only two actually failed against the pre-round-1 script. Every test is now labelled REGRESSION or CHARACTERISATION, each verified by direct replay against the named prior version rather than asserted -- 14 of 33 are genuine regression tests. Both of the review's own defeats (block-wide PATTERN silencing, the 50-line range) are reproduced as dedicated fixtures and confirmed caught; both are also confirmed to slip through the pre-round-2 tool unchanged. Claude-Session: https://claude.ai/code/session_017ZBxCCRM2ojnBupp3SBxV9

fix(tools): harden check_citations.py against its own self-poisoning bug

2026-08-19T13:01:18Z

Reproduced the defect: reintroducing the exact historical citation bug (pointing class-1's citation back at LT.txt:12459, the value a prior fix round corrected away from) made the tool report "147 citations checked, 0 look wrong". The mechanism was that the corrective comment documenting the old bug quotes the wrong historical value, and the checker pooled every quoted phrase from the whole surrounding comment block, so citing the wrong line matched the comment explaining why it was wrong. Four changes: 1. The word pool for a citation is now scoped to the entry(ies) it is attached to only -- never to quoted text elsewhere in the comment. This is the direct fix for the self-poisoning bug. 2. A citation whose pool has fewer than two distinctive words (Latin liturgical headings are short and stopword-heavy) cannot discriminate the right line from a wrong nearby one. Such a citation is now reported CANNOT VERIFY and fails the target, instead of silently passing. 3. The blanket +-2-line tolerance is gone. A bare "LT.txt:N" is checked at line N only; a heading that genuinely wraps must say so explicitly as "LT.txt:N-M". The allowance moves into the data, where it is visible. 4. The tool gets its own test suite, tools/test_check_citations.py, with a synthetic fixture covering: a correct citation, off-by-one and off-by-three mismatches, an explicit wrap range, a degenerate pool, a PATTERN-marked entry with no citation, and a dedicated regression test for the self-poisoning case itself. Wired into `dune test` via a new (rule (alias runtest) ...) in tools/dune (a plain (test ...) stanza cannot run a Python script), so it runs with the rest of the suite, not only as a `make` target. Added a --file/--lt-file override to check_citations.py so the tool (and its own tests) can point at a fixture without touching the real lang/la.ini or docs/research/LT.txt. Confirmed the "SKIPPED, exit 0" behaviour for a missing docs/research/LT.txt is unchanged. tools/__pycache__/ (a stray artefact of this script, previously untracked and ungitignored) is now in .gitignore. Measured against the current lang/la.ini (another task is still landing its sanctoral entries on this branch): 15 of 275 citations now look wrong and 42 more cannot be verified, both far above the 0 the unhardened tool reported. Not fixed here -- the data pass is separate, once the sanctoral entries land.

fix(lang): correct two wrong citation lines, add check-citations

2026-08-19T12:35:36Z

Two of la.ini's LT.txt: citations pointed at the wrong line -- the Latin itself was right, only the pinned line was wrong: - advent cited LT.txt:8631 ("Tempus Nativitatis"); the real "Tempus Adventus" heading is at 8609. - ef-christ-the-king and [rank]'s own citation both pointed near "Dominica ultima Octobris" (12459) when the text they actually quote, "D.NI NOSTRI JESU CHRISTI REGIS" and "I classis", sits two and three lines further down, at 12461 and 12462. ef-christmas-sunday-0 was marked PATTERN but LT.txt:8644 is the identical string verbatim -- relabelled as a direct citation, not constructed. Added tools/check_citations.py and `make check-citations`: for every LT.txt: citation outside a PATTERN block, confirms a +-2-line window around line n actually contains the Latin text the citation claims, rather than trusting each of the 38 citations by hand. Follows check-schema/check-templates' own precedent -- docs/ is gitignored, so the target prints SKIPPED loudly and exits 0 when docs/research/LT.txt is absent, never a silent pass. The checker's own teeth are proven three ways: replayed against the pre-fix file it independently re-derives both corrections above; a fresh mutation (redirecting one citation to an unrelated line) is caught and reverted; the fixed file passes clean, 147 citations checked, 0 wrong.

docs(tools): the lectio patch exporter cannot see temporal corrections

2026-08-18T13:23:36Z

It compares colitur's sanctoral data against lectio's calendar ini, so a correction to a temporal day -- computed in code on both sides, present in neither file -- is structurally invisible to it. Not hypothetical. The Good Friday colour fix (RG 128(b) + RG 132, v0.4.0) was missed entirely by this tool when the eight-field patch was produced, and surfaced only by running clectio at the bottom of the chain and diffing its output against colitur date by date. The tool narrows the search; it does not close it.

feat(tools): export the lectio EF correction patch

2026-08-18T10:57:18Z

colitur has been downstream of lectio since its data was bootstrapped from it. These are the fields where that relationship should now invert: eight changes, each annotated with the colitur allow-list entry carrying its rubric. Deliberately a PATCH and not a regeneration. colitur carries 205 of the 327 Polish names lectio ships, so regenerating lectio's ini from colitur would silently drop 122 of them; colitur also uses a different slug vocabulary for Passion/Holy week and different Paschaltide week numbering. What colitur is authoritative for is the adjudicated fields, and only those are emitted. Comparing the two datasets field by field also caught a counting error in colitur's own register that no assertion in this project was watching for: six sanctoral entries carry subject = Lord, not the four recorded, because the regex that counted them truncated on the two longest Polish names. The RG 112(b) reachability measurement was re-run against all six and is unchanged at 0/0; both docs are corrected.

fix(lectionary): the week after Trinity says its own assigned Mass

2026-08-17T19:12:35Z

RG 299 located (scan1:1096-1098), which does two things at once. First it closes a standing gap: the ferial-Mass selection rule was recorded in the register as unconfirmed against the primary source, and step 3 said so in its own comment -- "lectio's own observed behaviour, not a confirmed Missal citation". It is confirmed now. "In reliquis feriis dicitur Missa dominicae praecedentis, nisi a rubricis aliter provisum sit", restated in the propers as "...nisi propria Missa assignetur". Second, that escape clause splits M26 shape 2 in half, which is why the shape was never resolvable as one thing. Shape 2(a), the week after Trinity, is fixed. Trinity is the temporal identity of its own Sunday, so a plain walkback lands on Trinity's Mass -- but the Missal prints a named formula for the following week, "Missa dominicae I post Pentecosten" (scan1:21758), precisely because the feast has taken that Sunday's own. Those ferias now resolve at step 2 from it and never reach the walkback. Shape 2(b), Christ the King's week, turns out not to be a bug at all. No proper Mass is assigned there -- checked directly, no rubric precedes that Mass heading (scan1:39163ff) -- so RG 299's default governs and "the Mass of the preceding Sunday" genuinely is Christ the King's own, which is what colitur emits and missalemeum does not. Kept open only because no rubric specific to that week was located either way; the verdict is now argued from the primary text rather than resting on oracle disagreement. No Thursday in the slug list: Corpus Christi is Easter+60, which IS that week's Thursday every year, so the ferial slug never exists. Found by the bootstrap's own reachability guard refusing to emit a key no Temporal_ef slug can match, not reasoned out in advance. Allow-lists: C35 opened (69 rows, verdict colitur -- lectio still walks back); C17 narrowed 406 to 397, nine Saturdays moving to C35 because they carry both a citation and a colour difference, with C35 gated to require the citation so a colour-only Saturday still falls to C17; M26 narrowed 30 to 27; M18 widened 352 to 355, the same three days keeping only their observed-identity difference. Register: section 6.11.

test(oracle): a second oracle year, 2038, closing step 4's blind spot

2026-08-17T14:00:23Z

Register section 6.7 recorded that step 4 of the reading chain -- the Common route -- had no external witness of any kind, and proved it by mutation: corrupting a Common citation left both the 16801-day differential and the 730-day oracle green. This closes that. 2038 is the only year in 2005-2050 in which two of the five Common-routed saints are the observed office (Perpetua and Felicitas on 6 March, Frances of Rome on 9 March). 365 days were captured live, one request per day, zero failures. A separate fixture, not more rows on the existing one. sources.md already records that the live endpoint has drifted from lectio's archived 2026-2027 snapshot; merging two versions of one source behind a single set of expectations would make any future disagreement unattributable -- calendar or drift, no way to tell. Nothing in the 2038 comparison is checked against the older fixture, and its provenance file says so, along with the fact that a live capture is not reproducible byte-for-byte on demand the way a snapshot-derived one is. 338 of 365 days match. 27 differ, every one of them in a named, already-adjudicated class, none unexplained: 13 the BVM Saturday votive Mass (M26 shape 1), 7 colour (M21), 2 Judith (M28), 2 Christ the King's week (M26 shape 2b), 1 Christmas multi-Mass (M27), 1 Perpetua's Common, 1 new. Each was decided in a different year, so 2038 re-confirms them independently -- the point of a second window is not new rulings but evidence the old ones are not artefacts of their own year. Two findings came out of it. Perpetua and Felicitas: missalemeum serves the Common of Virgins and calls them "Virgins and Martyrs" in its own oration, where the Missal directs "Missa Me exspectaverunt, de Communi non Virginum I loco" (scan1:27634-27635). Both Commons share the Introit Ps 118:95-96, which is exactly why the calendarium qualifies its direction, and both women were mothers. Verdict colitur; the mechanism is located on missalemeum's side, not merely asserted. Passion Tuesday: the Missal prints "Dan. 14, 27 et 28-42" (scan1:11106) and colitur reproduces that two-part form where missalemeum collapses it to 27-42. Same verses; the convention is deliberate, appearing also in the Seven Sorrows and the Common of Non-Virgins. Verdict colitur, cosmetic. Citations are notation-normalised before comparing, as layer 3 already does: Ecclus/Sir, Joann/John, Luc/Luke, Matth/Matt. Each pair was added because a real row needed it, and no target contains its own source as a substring, so the set is idempotent. Not compared, stated rather than left to be found: commemorations and observed-identity. That machinery is built around a date-literal 28-entry allow-list specific to 2026-2027, and re-deriving it for a second year is its own task. 2038 compares rank, colour, Epistle, Gospel. The extractor's day count was hardcoded to 730, which silently forbade any other window. It is now a parameter defaulting to 730, so the existing documented command keeps its guard and a partial fetch still fails loudly instead of producing a short fixture that passes a comparison it never ran. Teeth, by re-running section 6.7's own experiment: corrupting the Common of Non-Virgins II now reddens four tests, two of them external-oracle, where the same mutation previously reddened neither oracle layer. Residual: isidore-of-seville is still unwitnessed and needs 2035 or 2046; gregory-the-great and patrick are never the observed office in any year 2005-2050, so no fixture in that range can reach them. Register section 6.8 and 6.9.

test(oracle): fix round 1 -- C30 disclosure, RG 17(a) misattribution, M26 shapes

2026-08-17T11:02:39Z

Three Important findings from coordinator review of 8d1eb25, all fixed. 1. C30's own note claimed its 91-row population "is C25's OWN former population, before this task narrowed it" -- false. Independently re-derived against the real resolver (pre-fix vs post-fix binary) and the lectio fixture: only 44 of the 91 rows were ever in C25 (Wed 6/ Thu 19/Fri 19, matching C25's own fix-round-3 breakdown exactly); the other 47 (all of Monday/Tuesday, plus part of Wednesday/Friday) previously matched lectio outright -- both engines independently landing on the same wrong value -- and are newly allow-listed only because colitur is now right. Rewritten to the same "both engines silently agreed on the wrong answer" pattern C33/C34 already used. 2. "Diebus ferialibus" is not RG 17(a) -- RG 17(a) states only the day-identity privilege; the ferial-Mass rubrics are Mass-propers rubrics, a distinction the register's own §6.3 already drew before this task's first pass blurred it in code/allow-list comments. A further slip: C31/C32 called their own rubric "RG 17(a)'s own next/ third sentence" when it is a different rubric block ~230 lines away, in Holy Family's (not Holy Name's) own feast propers. Fixed the attribution chain throughout tools/bootstrap_lectionary.ml, data/ef/expected-divergences.sexp's C29-C34, and (gitignored, outside this commit) docs/research/rules-register.md -- closed the §6.3 park, added §6.5 recording all four citations with corrected attribution. Quotes and line numbers were already right; only the chain of custody to RG 17(a) was wrong. 3. M26's own 30 rows classify into two nameable shapes, not "three confirmed shapes" with an unexplained 23-row residue that was in fact empty: 21 rows are RG 78's own already-documented BVM votive-Mass gap; 9 rows are a second, distinct gap -- a week whose own Sunday was displaced by a movable named feast (Trinity->Corpus Christi, "Missa Dominicae I post Pentecosten"; Christ the King), where the Missal requires the displaced Sunday's own Mass to keep informing that week's ferias but step 3 walks back to the displacing feast's citation instead -- proven directly by colitur's own citation on the Christ- the-King rows being literally that feast's own Gospel. verdict open unchanged (not fixed here); the shape is now named instead of buried. Minors: section count corrected 1458->1456 (four places); "exactly one candidate, zero ambiguous" corrected to "correctness rests on ordering, not uniqueness" (2 of 1456 sections genuinely yield two candidates, Holy Saturday's own Lectio, both years); the stale "nothing in layers 3-5 compares citations" claim corrected (only layer 5 still doesn't); M18's note corrected (five new entries not six; ~18 of its remaining 352 rows only agree now because of this task's own fixes, not "already" before it); "five" corrected to "six" otherwise-unoccupied Saturdays in bootstrap_lectionary.ml; scan2 quotes softened where OCR-garbled. Also done outside the git-tracked files: the step-4 (Commons) coverage gap is now recorded in data/ef/expected-divergences-missalemeum.sexp's own header, not only the commit body and the (gitignored) report. dune test: 358 tests green. COLITUR_EXHAUSTIVE_SWEEP=1 dune test --force: 359 tests green, 95.7s.

test(oracle): compare reading citations against missalemeum

2026-08-17T10:28:03Z

Layer 4 (test_oracle.ml) now extracts and compares the Epistle/Gospel citation for every one of the 730 days in the 2026-2027 fixture, the same axis Task 8 already added to layer 3. tools/ extract_missalemeum_oracle.py gained a structural citation-span detector (the first "*...*"-wrapped span under 40 chars containing a chapter:verse-shaped digit pair), verified exhaustively over all 1 458 Lectio/Evangelium sections. 728 of 730 days are comparable; the 2 that are not (Good Friday, both years) have no such section in missalemeum's own JSON at all -- a structural limit on the oracle's own side (M23), not a colitur gap. 451 of 730 days differ and are all explained across 19 cited entries; 0 unexplained. Chain step 4 (the Commons) has no layer-4 witness: confirmed empirically (a scratch sweep, not merely re-asserted from Task 6's own report) that none of the 15 Commons/proper saints is ever the observed office in this window. The comparison surfaced three genuine colitur bugs in data/ef/lectionary.sexp, all fixed at the source in tools/bootstrap_lectionary.ml: 1. Holy Name of Jesus (ef-holy-name-sunday/ef-holy-name) carried a mis-derived citation, borrowed by analogy from an unrelated ini section ("ef-christmas-sunday-0", the Sunday-within-the-Octave's own Mass) on the wrong assumption the two share a Mass. Disproved by missalemeum's own extraction for both real Holy Name Sundays in its window, confirmed against the Missal itself, twice, both scans (Acts 4:8-12/Luke 2:21, not Gal 4:1-7/Luke 2:33-40). 2. RG 17(a)'s own "diebus ferialibus" rubric (immediately following the Holy Name heading, both scans) was never implemented: 2-5 January ferias repeat 1 January's Mass, 7-12 January ferias before Holy Family Sunday repeat Epiphany's, and Holy Family's own week has a named formula ("Missa Dominicae Primae post Epiphaniam"). All three fell to step 3's undifferentiated Sunday walkback instead. Fixed for three 5-weekday families, deliberately excluding Saturday from all three -- RG 78's BVM Saturday Office wins there unconditionally, confirmed against missalemeum's own title on this window's own such Saturdays. 3. Corpus Christi and Sacred Heart (both real, never-impeded I-class movable feasts) had no lectionary entry at all. Both engines independently fell back to Trinity Sunday's own citation and silently agreed on the wrong answer -- the "Holy Thursday violet in both" shape CLAUDE.md names for exactly this reason. Because lectionary.sexp is shared with layer 3 (the lectio differential, 2005-2050), these fixes changed colitur's output for the whole domain, not only 2026-2027. data/ef/expected-divergences.sexp and test_differential.ml needed matching updates: C1 narrowed 361->173 and C25 narrowed 57->13 (rows that now resolve correctly moved to new entries), C16's note corrected (count unchanged), and six new entries added (C29-C34, 397 rows total, all verdict colitur). test_lectionary.ml and test_lectionary_ef.ml's own pins were updated to match (entry count 131->148; one step-3 regression test re-pointed to a date that still genuinely exercises that step). One entry, M26 (30 rows, verdict open), is a genuine unfixed gap: colitur's step-3 "resume the preceding Sunday" fallback is its only ferial-Mass mechanism, and missalemeum's own selection is more granular in three confirmed ways (a rotating votive Mass of the BVM, a named weekly formula for at least one week, and later Time-after-Pentecost weeks this task's own investigation did not narrow to a single rule) -- the same "needs a season-keyed selection mechanism, new behaviour not new data" shape data/ef/expected-divergences.sexp's own C6/C25 and Task 6's own residual concern already name, out of this task's own scope to build. dune test: 358 tests green. COLITUR_EXHAUSTIVE_SWEEP=1 dune test --force: 359 tests green, 89.5s.

test(differential): fix round 3 -- two more Ember Saturdays, doc fixes

2026-08-15T02:48:59Z

Two Ember Saturdays shipped an apostle's Mass verbatim from lectio's own ini, the exact shape already found and fixed for Passion Tuesday (C26): ef-advent-ember-sat carried St Thomas the Apostle's Mass (Eph 2:19-22 / John 20:24-29, 21 December) instead of the real "Sabbato Quatuor Temporum Adventus" (2 Thess 2:1-8 / Luke 3:1-6); ef-september- ember-sat carried St Matthew's Mass (Ezek 1:10-14 / Matt 9:9-13, 21 September) instead of the real "Sabbato Quatuor Temporum septembris" (Heb 9:2-12 / Luke 13:6-17). Both verified against both primary-source scans independently. Both readings are explicitly labelled "Lectio Epistolae" in the Missal on both days, so neither needed the "last lesson before the Gospel" convention Holy Wednesday/Good Friday relied on -- a textual fact, not an editorial choice. Fixed at the source in tools/bootstrap_lectionary.ml: colitur_keys excludes both wrong ini sections; a new ember_saturday_corrections list hand-authors the real citations. New allow-list entries C27/C28 (40 rows each, not the naive 46 -- traced to the identical 6 civil years where 21 September and 21 December, always exactly 91 days apart, land on a Saturday and the fixed saint's feast wins outright). Both values pinned in test_lectionary.ml and mutation-tested. Also: C25's per-weekday breakdown was mislabelled (right multiset, rotated weekday labels) in both test_differential.ml and expected- divergences.sexp -- corrected to Wednesday 6/Thursday 19/Friday 19/Saturday 13, and the claim that this split came directly from the comparator's own output is narrowed to what's actually true (only the total, 57, does). C26's scan2 citation range was extended to include the Gospel (previously stopped after the Epistle). lib/kernel/liturgical_day.mli's "citations always empty until Plan 4" comment is now stale on this branch -- left untouched (Task 11's own documentation pass), noted in the task report so it isn't lost. dune test: 358 tests green; COLITUR_EXHAUSTIVE_SWEEP=1: 359 green.