colitur.git/tools/extract_missalemeum_oracle.py, branch v1.1.0

test(oracle): a second oracle year, 2038, closing step 4's blind spot

2026-08-17T14:00:23Z

Register section 6.7 recorded that step 4 of the reading chain -- the Common route -- had no external witness of any kind, and proved it by mutation: corrupting a Common citation left both the 16801-day differential and the 730-day oracle green. This closes that. 2038 is the only year in 2005-2050 in which two of the five Common-routed saints are the observed office (Perpetua and Felicitas on 6 March, Frances of Rome on 9 March). 365 days were captured live, one request per day, zero failures. A separate fixture, not more rows on the existing one. sources.md already records that the live endpoint has drifted from lectio's archived 2026-2027 snapshot; merging two versions of one source behind a single set of expectations would make any future disagreement unattributable -- calendar or drift, no way to tell. Nothing in the 2038 comparison is checked against the older fixture, and its provenance file says so, along with the fact that a live capture is not reproducible byte-for-byte on demand the way a snapshot-derived one is. 338 of 365 days match. 27 differ, every one of them in a named, already-adjudicated class, none unexplained: 13 the BVM Saturday votive Mass (M26 shape 1), 7 colour (M21), 2 Judith (M28), 2 Christ the King's week (M26 shape 2b), 1 Christmas multi-Mass (M27), 1 Perpetua's Common, 1 new. Each was decided in a different year, so 2038 re-confirms them independently -- the point of a second window is not new rulings but evidence the old ones are not artefacts of their own year. Two findings came out of it. Perpetua and Felicitas: missalemeum serves the Common of Virgins and calls them "Virgins and Martyrs" in its own oration, where the Missal directs "Missa Me exspectaverunt, de Communi non Virginum I loco" (scan1:27634-27635). Both Commons share the Introit Ps 118:95-96, which is exactly why the calendarium qualifies its direction, and both women were mothers. Verdict colitur; the mechanism is located on missalemeum's side, not merely asserted. Passion Tuesday: the Missal prints "Dan. 14, 27 et 28-42" (scan1:11106) and colitur reproduces that two-part form where missalemeum collapses it to 27-42. Same verses; the convention is deliberate, appearing also in the Seven Sorrows and the Common of Non-Virgins. Verdict colitur, cosmetic. Citations are notation-normalised before comparing, as layer 3 already does: Ecclus/Sir, Joann/John, Luc/Luke, Matth/Matt. Each pair was added because a real row needed it, and no target contains its own source as a substring, so the set is idempotent. Not compared, stated rather than left to be found: commemorations and observed-identity. That machinery is built around a date-literal 28-entry allow-list specific to 2026-2027, and re-deriving it for a second year is its own task. 2038 compares rank, colour, Epistle, Gospel. The extractor's day count was hardcoded to 730, which silently forbade any other window. It is now a parameter defaulting to 730, so the existing documented command keeps its guard and a partial fetch still fails loudly instead of producing a short fixture that passes a comparison it never ran. Teeth, by re-running section 6.7's own experiment: corrupting the Common of Non-Virgins II now reddens four tests, two of them external-oracle, where the same mutation previously reddened neither oracle layer. Residual: isidore-of-seville is still unwitnessed and needs 2035 or 2046; gregory-the-great and patrick are never the observed office in any year 2005-2050, so no fixture in that range can reach them. Register section 6.8 and 6.9.

test(oracle): fix round 1 -- C30 disclosure, RG 17(a) misattribution, M26 shapes

2026-08-17T11:02:39Z

Three Important findings from coordinator review of 8d1eb25, all fixed. 1. C30's own note claimed its 91-row population "is C25's OWN former population, before this task narrowed it" -- false. Independently re-derived against the real resolver (pre-fix vs post-fix binary) and the lectio fixture: only 44 of the 91 rows were ever in C25 (Wed 6/ Thu 19/Fri 19, matching C25's own fix-round-3 breakdown exactly); the other 47 (all of Monday/Tuesday, plus part of Wednesday/Friday) previously matched lectio outright -- both engines independently landing on the same wrong value -- and are newly allow-listed only because colitur is now right. Rewritten to the same "both engines silently agreed on the wrong answer" pattern C33/C34 already used. 2. "Diebus ferialibus" is not RG 17(a) -- RG 17(a) states only the day-identity privilege; the ferial-Mass rubrics are Mass-propers rubrics, a distinction the register's own §6.3 already drew before this task's first pass blurred it in code/allow-list comments. A further slip: C31/C32 called their own rubric "RG 17(a)'s own next/ third sentence" when it is a different rubric block ~230 lines away, in Holy Family's (not Holy Name's) own feast propers. Fixed the attribution chain throughout tools/bootstrap_lectionary.ml, data/ef/expected-divergences.sexp's C29-C34, and (gitignored, outside this commit) docs/research/rules-register.md -- closed the §6.3 park, added §6.5 recording all four citations with corrected attribution. Quotes and line numbers were already right; only the chain of custody to RG 17(a) was wrong. 3. M26's own 30 rows classify into two nameable shapes, not "three confirmed shapes" with an unexplained 23-row residue that was in fact empty: 21 rows are RG 78's own already-documented BVM votive-Mass gap; 9 rows are a second, distinct gap -- a week whose own Sunday was displaced by a movable named feast (Trinity->Corpus Christi, "Missa Dominicae I post Pentecosten"; Christ the King), where the Missal requires the displaced Sunday's own Mass to keep informing that week's ferias but step 3 walks back to the displacing feast's citation instead -- proven directly by colitur's own citation on the Christ- the-King rows being literally that feast's own Gospel. verdict open unchanged (not fixed here); the shape is now named instead of buried. Minors: section count corrected 1458->1456 (four places); "exactly one candidate, zero ambiguous" corrected to "correctness rests on ordering, not uniqueness" (2 of 1456 sections genuinely yield two candidates, Holy Saturday's own Lectio, both years); the stale "nothing in layers 3-5 compares citations" claim corrected (only layer 5 still doesn't); M18's note corrected (five new entries not six; ~18 of its remaining 352 rows only agree now because of this task's own fixes, not "already" before it); "five" corrected to "six" otherwise-unoccupied Saturdays in bootstrap_lectionary.ml; scan2 quotes softened where OCR-garbled. Also done outside the git-tracked files: the step-4 (Commons) coverage gap is now recorded in data/ef/expected-divergences-missalemeum.sexp's own header, not only the commit body and the (gitignored) report. dune test: 358 tests green. COLITUR_EXHAUSTIVE_SWEEP=1 dune test --force: 359 tests green, 95.7s.

test(oracle): compare reading citations against missalemeum

2026-08-17T10:28:03Z

Layer 4 (test_oracle.ml) now extracts and compares the Epistle/Gospel citation for every one of the 730 days in the 2026-2027 fixture, the same axis Task 8 already added to layer 3. tools/ extract_missalemeum_oracle.py gained a structural citation-span detector (the first "*...*"-wrapped span under 40 chars containing a chapter:verse-shaped digit pair), verified exhaustively over all 1 458 Lectio/Evangelium sections. 728 of 730 days are comparable; the 2 that are not (Good Friday, both years) have no such section in missalemeum's own JSON at all -- a structural limit on the oracle's own side (M23), not a colitur gap. 451 of 730 days differ and are all explained across 19 cited entries; 0 unexplained. Chain step 4 (the Commons) has no layer-4 witness: confirmed empirically (a scratch sweep, not merely re-asserted from Task 6's own report) that none of the 15 Commons/proper saints is ever the observed office in this window. The comparison surfaced three genuine colitur bugs in data/ef/lectionary.sexp, all fixed at the source in tools/bootstrap_lectionary.ml: 1. Holy Name of Jesus (ef-holy-name-sunday/ef-holy-name) carried a mis-derived citation, borrowed by analogy from an unrelated ini section ("ef-christmas-sunday-0", the Sunday-within-the-Octave's own Mass) on the wrong assumption the two share a Mass. Disproved by missalemeum's own extraction for both real Holy Name Sundays in its window, confirmed against the Missal itself, twice, both scans (Acts 4:8-12/Luke 2:21, not Gal 4:1-7/Luke 2:33-40). 2. RG 17(a)'s own "diebus ferialibus" rubric (immediately following the Holy Name heading, both scans) was never implemented: 2-5 January ferias repeat 1 January's Mass, 7-12 January ferias before Holy Family Sunday repeat Epiphany's, and Holy Family's own week has a named formula ("Missa Dominicae Primae post Epiphaniam"). All three fell to step 3's undifferentiated Sunday walkback instead. Fixed for three 5-weekday families, deliberately excluding Saturday from all three -- RG 78's BVM Saturday Office wins there unconditionally, confirmed against missalemeum's own title on this window's own such Saturdays. 3. Corpus Christi and Sacred Heart (both real, never-impeded I-class movable feasts) had no lectionary entry at all. Both engines independently fell back to Trinity Sunday's own citation and silently agreed on the wrong answer -- the "Holy Thursday violet in both" shape CLAUDE.md names for exactly this reason. Because lectionary.sexp is shared with layer 3 (the lectio differential, 2005-2050), these fixes changed colitur's output for the whole domain, not only 2026-2027. data/ef/expected-divergences.sexp and test_differential.ml needed matching updates: C1 narrowed 361->173 and C25 narrowed 57->13 (rows that now resolve correctly moved to new entries), C16's note corrected (count unchanged), and six new entries added (C29-C34, 397 rows total, all verdict colitur). test_lectionary.ml and test_lectionary_ef.ml's own pins were updated to match (entry count 131->148; one step-3 regression test re-pointed to a date that still genuinely exercises that step). One entry, M26 (30 rows, verdict open), is a genuine unfixed gap: colitur's step-3 "resume the preceding Sunday" fallback is its only ferial-Mass mechanism, and missalemeum's own selection is more granular in three confirmed ways (a rotating votive Mass of the BVM, a named weekly formula for at least one week, and later Time-after-Pentecost weeks this task's own investigation did not narrow to a single rule) -- the same "needs a season-keyed selection mechanism, new behaviour not new data" shape data/ef/expected-divergences.sexp's own C6/C25 and Task 6's own residual concern already name, out of this task's own scope to build. dune test: 358 tests green. COLITUR_EXHAUSTIVE_SWEEP=1 dune test --force: 359 tests green, 89.5s.

feat(test): oracle layer compares commemoration identity, not just count

2026-08-12T13:37:08Z

test_oracle.ml (validation layer 4) used to compare commemoration presence and List.length only -- cardinality, never which candidate is admitted. A reviewer proved this vacuous by reversing admit's dignity sort (the engine admitting the worst commemoration, an outright RG 111 violation) and finding every assertion in this file green across all 730 days. tools/extract_missalemeum_oracle.py now also emits each commemoration's own missalemeum id (field 9, commemoration_ids), parallel to the existing title field; fixture regenerated from the same underlying lectio snapshot (commit d7da4b0, re-verified, first 8 fields byte-identical), read from a scratch copy so ~/git/projects/lectio was never touched. SHA-256 pin and provenance updated. The mapping: colitur identifies a commemoration by slug, missalemeum by an English title string. No hand-built slug<->title table -- colitur's own Celebration.t.names already carries the same English text for every SANCTORAL-origin commemoration (bootstrapped from lectio, verified to match missalemeum's titles character-for-character). This resolves identity for every sanctoral-origin commemoration but cannot resolve a TEMPORAL-origin one (Rite_ef.Temporal_ef never sets a celebration name) -- that case is a new, distinct outcome (Comm_identity_unresolved), counted and allow-listed, never silently skipped: 19 of 227 non-empty-commemoration days over 2026-2027. Two genuine identity mismatches found and adjudicated (new allow-list entries M15-M17, one is the unresolved bucket): M16, 27 March 2026, the already-known unimplemented Seven Sorrows of Passion Friday office (verdict missalemeum); M17, 22 September 2027, the RG 113 same-band tie-break residual (verdict open, per the sibling admit fix). A new corroborating check cross-references missalemeum's own id-embedded rank against colitur's, split by status: Feast-status matches must agree exactly (none reachable in this 2-year window, checked regardless); Commemoration_only-status matches either genuinely agree or fit a known, pre-existing, already-registered convention gap (colitur's bootstrap rank vs missalemeum's own sentinel) -- not a title-matching defect. Perturbation test performed and reverted: reversing admit's dignity- based ordering (compare_precedence's real committed form uses band, not dignity -- this reproduces the historical defect shape directly) turns the oracle layer red: "no differences outside the cited allow-list" fails with an unexplained commemoration-identity-mismatch on 2026-09-23 (colitur admits "St. Thecla" where the rubric and missalemeum both require "St. Linus"). Reverted; diff against the prior commit confirms no residue. CLAUDE.md's "know what each layer cannot see" section updated: the commemoration-identity gap it documented as open is now closed, with the mapping's limits and the RG 113 tie-break finding summarised. 271 -> 273 tests (274 with the exhaustive sweep), all green.

test: oracle vs missalemeum 2026-2027; audit the sanctoral

2026-08-12T02:36:36Z

Validation layer 4: an oracle harness against missalemeum (Divinum Officium data), independent of the lectio bootstrap chain colitur's own sanctoral data comes from -- the only layer that can catch an error inherited from that bootstrap, and the only one that can validate commemorations at all (the lectio differential explicitly excludes them, per its own header comment). tools/extract_missalemeum_oracle.py shapes the fixture from lectio's sources/snapshot.tar.gz outside the test (no JSON library in this project's frozen deps, same reasoning test_differential.ml's own fixture already documents). test/fixtures/missalemeum-ef-2026-2027 .txt (730 days, SHA-256 pinned and asserted) + its own .provenance note record exactly how to regenerate it. test_oracle.ml compares three axes the oracle actually supports: rank, colour (SET MEMBERSHIP -- 14 of 730 days carry two colours, e.g. rose+violet on Gaudete/Laetare, which independently vindicates this project's own rose reading against lectio's violet-only one, recorded in the register), and commemoration presence/count. Slug identity is deliberately out of scope (needs a title->slug mapping, the data audit's own business, not the automated comparator's). Of 730 days, 688 matched cleanly outright. The remaining 42 are all named in data/ef/expected-divergences-missalemeum.sexp (14 cited entries, M1-M14): most are genuine primary-source-confirmed findings this task adjudicated and fixed in the two preceding commits (RG 33, RG 109/111, Holy Thursday's colour); the rest are real, cited, deferred feature/data gaps (RG 91 entry 27's BVM-Saturday office, RG 110's inseparable Peter/Paul commemoration, four sanctoral entries missing from lectio's own source) or genuine oracle-side artifacts -- honestly verdicted against whichever side this task's own primary-source research actually backs, never defaulted to colitur. One entry (M13, St Joseph vs the Friday of Passion Week 2027) is verdict open: adjudicated as unresolved after real search effort, not guessed past. The data audit: every sanctoral entry the comparison flagged was hand-checked against the 1962 calendarium, plus a 20-entry deterministic random control sample (seed 20260812) drawn independently of the flagged set. The control sample caught two entries (benedict, frances-rome) marked Commemoration_only in the bootstrapped data when the primary calendarium lists them as plain III-class feasts with their own Office -- traced to lectio's own source, not fixable here, and reported as a signal (10% of a random sample) rather than a blanket claim. Coverage recorded honestly in docs/research/rules-register.md's own three buckets: confirmed by oracle (200/322), confirmed by hand (23/322, 2 of them wrong), unverified (115/322) -- the unverified bucket stated explicitly rather than left implicit. Harness teeth demonstrated and reverted (not committed): a fixture rank/colour edit on a previously-clean day fails both the checksum pin and the no-unexplained-differences assertion independently; an expected_rows drift on the allow-list fails the citation-count assertion. Both captured with their exact failure messages, both reverted before this commit.