diff options
| author | Lukasz Kasprzak <lukas@labunix.xyz> | 2026-08-12 15:37:08 +0200 |
|---|---|---|
| committer | Lukasz Kasprzak <lukas@labunix.xyz> | 2026-08-12 15:37:08 +0200 |
| commit | 76298db9b0fb5409bcfd4276b642f69f4da530e9 (patch) | |
| tree | 79b0f5c5e38a07094f1ca2449bf21b1528af9709 /test/test_oracle.ml | |
| parent | ea22ad2bde211998e6719fd5fb76233571a48383 (diff) | |
| download | colitur-76298db9b0fb5409bcfd4276b642f69f4da530e9.tar.gz colitur-76298db9b0fb5409bcfd4276b642f69f4da530e9.zip | |
feat(test): oracle layer compares commemoration identity, not just count
test_oracle.ml (validation layer 4) used to compare commemoration
presence and List.length only -- cardinality, never which candidate is
admitted. A reviewer proved this vacuous by reversing admit's dignity
sort (the engine admitting the worst commemoration, an outright RG 111
violation) and finding every assertion in this file green across all
730 days.
tools/extract_missalemeum_oracle.py now also emits each commemoration's
own missalemeum id (field 9, commemoration_ids), parallel to the
existing title field; fixture regenerated from the same underlying
lectio snapshot (commit d7da4b0, re-verified, first 8 fields
byte-identical), read from a scratch copy so ~/git/projects/lectio was
never touched. SHA-256 pin and provenance updated.
The mapping: colitur identifies a commemoration by slug, missalemeum by
an English title string. No hand-built slug<->title table -- colitur's
own Celebration.t.names already carries the same English text for every
SANCTORAL-origin commemoration (bootstrapped from lectio, verified to
match missalemeum's titles character-for-character). This resolves
identity for every sanctoral-origin commemoration but cannot resolve a
TEMPORAL-origin one (Rite_ef.Temporal_ef never sets a celebration name)
-- that case is a new, distinct outcome (Comm_identity_unresolved),
counted and allow-listed, never silently skipped: 19 of 227
non-empty-commemoration days over 2026-2027.
Two genuine identity mismatches found and adjudicated (new allow-list
entries M15-M17, one is the unresolved bucket): M16, 27 March 2026, the
already-known unimplemented Seven Sorrows of Passion Friday office
(verdict missalemeum); M17, 22 September 2027, the RG 113 same-band
tie-break residual (verdict open, per the sibling admit fix). A new
corroborating check cross-references missalemeum's own id-embedded rank
against colitur's, split by status: Feast-status matches must agree
exactly (none reachable in this 2-year window, checked regardless);
Commemoration_only-status matches either genuinely agree or fit a known,
pre-existing, already-registered convention gap (colitur's bootstrap
rank vs missalemeum's own sentinel) -- not a title-matching defect.
Perturbation test performed and reverted: reversing admit's dignity-
based ordering (compare_precedence's real committed form uses band, not
dignity -- this reproduces the historical defect shape directly) turns
the oracle layer red: "no differences outside the cited allow-list"
fails with an unexplained commemoration-identity-mismatch on 2026-09-23
(colitur admits "St. Thecla" where the rubric and missalemeum both
require "St. Linus"). Reverted; diff against the prior commit confirms
no residue.
CLAUDE.md's "know what each layer cannot see" section updated: the
commemoration-identity gap it documented as open is now closed, with
the mapping's limits and the RG 113 tie-break finding summarised.
271 -> 273 tests (274 with the exhaustive sweep), all green.
Diffstat (limited to 'test/test_oracle.ml')
| -rw-r--r-- | test/test_oracle.ml | 383 |
1 files changed, 351 insertions, 32 deletions
diff --git a/test/test_oracle.ml b/test/test_oracle.ml index 0aed1b9..07a5634 100644 --- a/test/test_oracle.ml +++ b/test/test_oracle.ml @@ -14,11 +14,10 @@ that file's own limit 1). missalemeum is INDEPENDENT of colitur's bootstrap chain, and its JSON carries a real admitted-commemorations array (its own RG 111-shaped selection, not lectio's rejects) -- so this - file compares THREE axes the lectio differential cannot touch at all: - rank, colour (as SET membership, see below), and commemoration presence/ - count. Slug IDENTITY is deliberately NOT compared (the brief: needs a - title->slug mapping, which is the data audit's business -- see the task - report -- not this automated comparator's). + file compares axes the lectio differential cannot touch at all: rank, + colour (as SET membership, see below), commemoration presence, count, + and (CORRECTED 2026-08-12, Task B, branch ef-rg16a -- see below) IDENTITY + -- WHICH commemoration is admitted, not only how many. *** THE FIXTURE'S OWN SHAPE (fixtures/missalemeum-ef-2026-2027.txt, fixtures/missalemeum-ef-2026-2027.provenance, tools/ @@ -29,7 +28,59 @@ between the id-embedded rank and info.rank itself (task brief's own premise was wrong here; the brief's extraction note has been corrected). This file, like the extraction script, reads ONLY info.rank and - info.colors, never parses anything out of info.id. + info.colors for the DAY's own rank/colour, never parses anything out of + info.id for that purpose. + + *** COMMEMORATION IDENTITY (added 2026-08-12, Task B, branch ef-rg16a) + -- CORRECTING the previous header's own "deliberately NOT compared" + claim *** + + The gap this correction closes: layer 4 used to compare commemoration + PRESENCE and List.length ONLY -- cardinality, never identity. A reviewer + proved this was vacuous by reversing admit's own dignity/precedence sort + (Rite_ef.Precedence_ef, so the engine admits the WORST candidate, an + outright RG 111 violation) and finding every assertion in this file + still green across all 730 days (see the perturbation record in the task + report, reproduced and reverted, not left in the tree). This file now + also compares WHICH commemoration is admitted, using two independent + signals from missalemeum's own JSON: the commemoration's title text + (English, exact string) and, for a sancti-origin commemoration, its own + [id]'s embedded rank digit (used only as a corroborating check, never as + the primary signal -- see [Comm_identity] below). + + THE MAPPING, AND EXACTLY WHAT IT CANNOT RESOLVE: missalemeum identifies + a commemoration by an English TITLE string; colitur identifies one by a + SLUG. There is no hand-built slug<->title lookup table here -- colitur's + own {!Celebration.t}.names already carries an English name for every + SANCTORAL-origin celebration (bootstrapped from lectio's own English + titles, which verifiably match missalemeum's titles character-for- + character on every checked row), so the comparison reads that name + directly off colitur's own resolved output, the same field + test_differential.ml already treats as authoritative for OF/EF display + text. This resolves identity for every SANCTORAL-origin commemoration. + It CANNOT resolve a TEMPORAL-origin commemoration (an impeded feria, a + privileged Sunday, an Ember/Rogation day, etc.): {!Rite_ef.Temporal_ef}'s + own [build] never sets a celebration's [names] field (rite_ef/ + temporal_ef.ml has no [en "..."] string anywhere in it), so every such + candidate's own title is empty on colitur's side -- there is nothing to + compare against missalemeum's title text, and no principled way to + invent one without duplicating missalemeum's own English-prose + convention ("Feria III after the I Sunday of Lent") as a second, + unverified source of truth. THIS CASE IS NEVER SILENTLY SKIPPED: see + [Comm_identity_unresolved] below -- it is counted, reported, and must be + named in the allow-list like any other difference, exactly the discipline + the brief demands ("a day whose commemoration cannot be matched must be + visibly counted and reported, not quietly passed"). + + Identity is only even ATTEMPTED once presence and count already agree + (both non-empty, same length) -- a cardinality mismatch is already + [Comm_presence]/[Comm_count]'s own business (see [diff_fields] below), + and comparing SETS of different size would just restate that mismatch + under a third name, not add information. Compared as a SORTED MULTISET + of titles, not a sequence: missalemeum's own [commemorations] array order + is not independently verified against RG 113's own "commemoratio de + Tempore fit primo loco" ordering rule here, so this axis proves WHICH + commemorations are admitted, not what order they are listed in. info.colors is an ARRAY -- 14 of 730 days carry two colours (rose+violet on Gaudete/Laetare per RG 131's indult, red+violet on Palm Sunday, black+ @@ -56,14 +107,19 @@ entries; RG 110's inseparable-Peter/Paul commemoration is unimplemented code, a real feature this task did not build) -- honestly verdicted [missalemeum] (colitur is short a feature or a row, not right), never - silently absorbed as if colitur were correct. TWO entries (M11 and - M13) are [verdict open] -- CORRECTED, final fix wave, item 7: this - comment previously said "one entry (M13)", missing M11, whose own - verdict was changed from [colitur] to [open] in fix round 1 (see M11's - own entry below for why) but this summary was never updated to match. - Both are adjudicated as unresolved, not resolved either way -- the - brief's own explicit permission ("say so as an open item") used for - real, not defaulted past. See the task report for every entry's full + silently absorbed as if colitur were correct. THREE entries (M11, M13 + and, since Task B/ef-rg16a, M17) are [verdict open] -- CORRECTED, final + fix wave, item 7: this comment previously said "one entry (M13)", + missing M11, whose own verdict was changed from [colitur] to [open] in + fix round 1 (see M11's own entry below for why) but this summary was + never updated to match; M17 (the RG 113 same-band tie-break residual, + register §6.1) is a genuinely NEW open item, not a stale-comment fix. + ONE entry (M15) carries its own fourth verdict, [unresolvable] -- not a + rubric dispute or a data gap either engine is wrong about, but a LIMIT + of this comparator itself (see M15's own entry). All are adjudicated as + unresolved/unresolvable, not resolved either way -- the brief's own + explicit permission ("say so as an open item") used for real, not + defaulted past. See the task report for every entry's full reasoning and primary-source citation. *) module Cal = Colitur_kernel.Calendar @@ -74,6 +130,8 @@ module Slug = Colitur_kernel.Slug module Date = Colitur_kernel.Date module Cel = Colitur_kernel.Celebration module Colour = Colitur_kernel.Colour +module Names = Colitur_kernel.Names +module Lang = Colitur_kernel.Lang module V = Rite_ef.Vocab_ef (* Same relative paths test_differential.ml uses: dune test runs from @@ -88,7 +146,16 @@ let allow_list_path = "../data/ef/expected-divergences-missalemeum.sexp" same reason test_differential.ml's own [fixture_sha256] is: a hand-edit or partial re-extraction would otherwise silently turn the oracle into an unlabelled snapshot of whatever someone last ran. *) -let fixture_sha256 = "718eff1a03fa09c3927ef18d19fe7921c2d4d7611890c22566058b81fd7cc5ea" +(* CORRECTED 2026-08-12, Task B (branch ef-rg16a): field 9 + (commemoration_ids) appended to the fixture (tools/ + extract_missalemeum_oracle.py's own header) so this file can compare + commemoration IDENTITY, not merely presence/count -- see this file's own + header comment, rewritten below. Regenerated from the SAME underlying + snapshot (fixtures/missalemeum-ef-2026-2027.provenance: still lectio + commit d7da4b0, re-verified, first 8 fields byte-identical) -- only field + 9 is new content, so this digest changes but the fixture's own data does + not. *) +let fixture_sha256 = "4162e79d7a0bc03b81a34ce6cd3b074e2d32570217759bc3b4058ada75ea8781" (* Identical technique to test_differential.ml's own [sha256_of_file] (that file's own comment explains why: shelling out to [sha256sum] @@ -143,6 +210,7 @@ type oracle_row = { o_colours : char list; o_title : string; o_commemorations : string list; + o_commemoration_ids : string list; (** parallel to [o_commemorations], see field 9's own doc above *) o_displaced : string list; } @@ -150,16 +218,27 @@ let explode s = List.init (String.length s) (String.get s) let split_list_field f = if f = "-" then [] else String.split_on_char ';' f +(* A commemoration id's own trailing "rank" field, e.g. "sancti:01-05:4:r" -> + [Some 4] -- the SECOND, corroborating identity signal this file's own + header describes (test_identity_rank_corroboration, near the bottom of + this file). [None] for any id that does not have exactly 4 ':'-separated + parts (defensive; every id checked so far does). *) +let id_rank id = + match String.split_on_char ':' id with [ _; _; rank; _ ] -> int_of_string_opt rank | _ -> None + +let is_sancti_id id = String.length id >= 7 && String.sub id 0 7 = "sancti:" + let oracle_row_of_line line = match String.split_on_char '|' line with - | [ date; rank; colours; title; _tempora; commemorations; displaced; _n_masses ] -> + | [ date; rank; colours; title; _tempora; commemorations; displaced; _n_masses; commemoration_ids ] -> { o_date = date; o_rank = int_of_string rank; o_colours = explode colours; o_title = title; o_commemorations = split_list_field commemorations; + o_commemoration_ids = split_list_field commemoration_ids; o_displaced = split_list_field displaced } - | _ -> Alcotest.failf "malformed fixture line (expected 8 '|'-separated fields): %S" line + | _ -> Alcotest.failf "malformed fixture line (expected 9 '|'-separated fields): %S" line let read_lines path = let ic = open_in path in @@ -183,7 +262,20 @@ let oracle_rows () = List.map oracle_row_of_line (read_lines fixture_path) (* year, which straddles two civil years). *) (* ---------------------------------------------------------------------- *) -type colitur_row = { c_date : string; c_rank : int; c_colour : char; c_commemorations : string list } +(* [c_commemorations]: one entry per admitted commemoration, carrying its + own slug, RG 8 rank, STATUS, AND its English name if colitur's own + {!Celebration.t}.names has one -- [None] for a TEMPORAL-origin + candidate, which [Rite_ef.Temporal_ef] never names (this file's own + header, "THE MAPPING, AND EXACTLY WHAT IT CANNOT RESOLVE"). The [None] + case is what [Comm_identity_unresolved] below reads; [status] is read by + [test_identity_rank_corroboration]'s own scope guard, see its comment. *) +type colitur_row = { + c_date : string; + c_rank : int; + c_colour : char; + c_commemorations : (string * int * Cel.status * string option) list; + (** slug, rank, status, English name *) +} let rank_to_int = function V.Class1 -> 1 | V.Class2 -> 2 | V.Class3 -> 3 | V.Class4 -> 4 @@ -195,6 +287,8 @@ let colour_to_char = function | Colour.Rose -> 'p' | Colour.Black -> 'b' +let en = Lang.of_string_exn "en" + let colitur_rows_2026_2027 () = let layer = real_layer () in let by_rata : (int, (V.season, V.rank) LD.t) Hashtbl.t = Hashtbl.create 800 in @@ -210,14 +304,17 @@ let colitur_rows_2026_2027 () = (match Hashtbl.find_opt by_rata (Date.to_rata !d) with | Some day -> let cel = day.LD.observed in - let commemoration_slugs = - List.map (fun (c, _) -> Slug.to_string c.Cel.slug) day.LD.commemorations + let commemorations = + List.map + (fun (c, _) -> + (Slug.to_string c.Cel.slug, rank_to_int c.Cel.rank, c.Cel.status, Names.find c.Cel.names en)) + day.LD.commemorations in rows := { c_date = Date.to_iso8601 day.LD.date; c_rank = rank_to_int cel.Cel.rank; c_colour = colour_to_char cel.Cel.colour; - c_commemorations = commemoration_slugs + c_commemorations = commemorations } :: !rows | None -> Alcotest.failf "internal error: no resolved day for %s" (Date.to_iso8601 !d)); @@ -226,18 +323,39 @@ let colitur_rows_2026_2027 () = List.rev !rows (* ---------------------------------------------------------------------- *) -(* Field-diff computation: the four axes the oracle actually supports *) -(* (identity-by-slug is deliberately out of scope, see this file's own *) -(* header). *) +(* Field-diff computation: the axes the oracle supports, RANK/COLOUR/ *) +(* PRESENCE/COUNT as before, plus (Task B) IDENTITY -- see this file's own *) +(* header for the mapping and its exact limits. *) (* ---------------------------------------------------------------------- *) -type field = Rank | Colour_f | Comm_presence | Comm_count +type field = Rank | Colour_f | Comm_presence | Comm_count | Comm_identity_mismatch | Comm_identity_unresolved let field_name = function | Rank -> "rank" | Colour_f -> "colour" | Comm_presence -> "commemoration-presence" | Comm_count -> "commemoration-count" + | Comm_identity_mismatch -> "commemoration-identity-mismatch" + | Comm_identity_unresolved -> "commemoration-identity-unresolved" + +(* Identity (Task B): only even attempted once presence AND count already + agree (both non-empty, same length) -- see this file's own header, + "Identity is only even ATTEMPTED...". Two outcomes beyond a clean match: + [Comm_identity_unresolved] when ANY of colitur's own admitted + commemorations carries no English name to compare (a TEMPORAL-origin + candidate -- counted and reported, never silently skipped, per the + brief); [Comm_identity_mismatch] when every name IS resolvable but the + SORTED MULTISET of titles still disagrees with missalemeum's own. *) +let identity_diff (c : colitur_row) (o : oracle_row) = + let c_has = c.c_commemorations <> [] and o_has = o.o_commemorations <> [] in + if not (c_has && o_has) || List.length c.c_commemorations <> List.length o.o_commemorations then None + else + let unresolved = List.exists (fun (_, _, _, name) -> name = None) c.c_commemorations in + if unresolved then Some Comm_identity_unresolved + else + let c_titles = List.sort compare (List.filter_map (fun (_, _, _, n) -> n) c.c_commemorations) in + let o_titles = List.sort compare o.o_commemorations in + if c_titles = o_titles then None else Some Comm_identity_mismatch let diff_fields (c : colitur_row) (o : oracle_row) = let c_has = c.c_commemorations <> [] and o_has = o.o_commemorations <> [] in @@ -252,7 +370,8 @@ let diff_fields (c : colitur_row) (o : oracle_row) = same disagreement under a second name. *) (if c_has && o_has && List.length c.c_commemorations <> List.length o.o_commemorations then Some Comm_count - else None) + else None); + identity_diff c o ] (* ---------------------------------------------------------------------- *) @@ -491,6 +610,76 @@ let m12_dates = [ "2027-02-22" ] item. *) let m13_dates = [ "2027-03-19" ] +(* M15 -- Task B (branch ef-rg16a), NOT a rubric dispute and NOT a data gap + either engine is wrong about: a genuine LIMIT of this comparator, honestly + counted rather than silently passed (the brief's own explicit + instruction: "a day whose commemoration cannot be matched must be + visibly counted and reported, not quietly passed"). On each of these 19 + rows, colitur's own admitted commemoration is TEMPORAL-origin (an + impeded Advent/Lent feria) -- Rite_ef.Temporal_ef's own [build] + (temporal_ef.ml) never sets a celebration's [names] field, so there is no + English string on colitur's side to compare against missalemeum's title + text at all (this file's own header, "THE MAPPING, AND EXACTLY WHAT IT + CANNOT RESOLVE"). Every one of these 19 oracle titles reads as a plain + prose description of the SAME temporal ferial slot colitur independently + computes for that date (e.g. 2 Dec 2026: oracle "Feria IV after I Sunday + of Advent" against colitur's own [ef-advent-1-wednesday]; 4 Dec 2027: + oracle "Sabbato after I Sunday of Advent" against [ef-advent-1-saturday]) + -- suggestive corroboration, recorded here for a human reader, but NOT + proof: this comparator has no mechanism to verify English prose against + an opaque slug, so it does not claim these as verified matches, only as + unresolved. Fixing this at the ROOT (giving every temporal-cycle + candidate an English name) is a data/lectionary-bootstrap task, not + something this comparator can do for itself -- register §6, "RG 113 + tie-break" section's own neighbour, records it as an open item. *) +let m15_dates = + [ "2026-02-24"; "2026-03-19"; "2026-03-25"; "2026-12-02"; "2026-12-03"; "2026-12-04"; + "2026-12-07"; "2026-12-11"; "2026-12-21"; "2027-02-24"; "2027-12-02"; "2027-12-03"; + "2027-12-04"; "2027-12-06"; "2027-12-07"; "2027-12-11"; "2027-12-13"; "2027-12-16"; + "2027-12-21" ] + +(* M16 -- Task B: a genuine IDENTITY mismatch, both sides otherwise agreeing + on rank/colour/count/presence. 27 March 2026 (Friday of Passion Week): + colitur admits "St. John Damascene" (a real Class3 universal feast that + genuinely falls that day); missalemeum shows "For Our Lady of the Seven + Sorrows" instead. The SAME standing gap M13 above already names ("the + 1962 calendarium's own March table carries... 'Feria VI post dominicam I + Passionis: Commemoratio septem Dolorum B. Mariae Virg.'... a fixed + commemoration... EVERY year") -- confirmed real, and confirmed + UNIMPLEMENTED in this codebase (no candidate for it is ever constructed + in temporal_ef.ml), so colitur has nothing to admit in the Seven + Sorrows' place and falls back to whichever ordinary Class3 saint + actually wins that Friday instead. Verdict missalemeum: colitur is + missing a real, primary-cited office, not differently opinioned. + Register §6 open item (same one M13/M16's own primary-source note + already tracks). Only 2026 shows here -- 2027's Friday of Passion Week + IS 19 March, M13's own date, where the identity axis cannot even be + reached (M13's own rank/colour mismatch already excludes that day from + count-matched identity comparison). *) +let m16_dates = [ "2026-03-27" ] + +(* M17 -- Task B: the SAME-band residual tie-break (docs/research/rules- + register.md §4's own "RG 113" entry, and §6.1's "RG 113 same-band + residual" record) made visible for the first time by identity + comparison. 22 September 2027 (September Ember Wednesday): colitur + admits "St. Maurice and Companions, Martyrs" (Commemoration_only, + Class3); missalemeum shows "St. Thomas of Villanova" (Feast, Class3) + instead -- both land on {!Precedence_ef.band} entry 24 (III-class + universal feasts), a genuine tie RG 113 gives no further instruction for + (its own text only reaches "servetur ordo tabellae praecedentiae", the + TABLE order; nothing in the primary text breaks a tie WITHIN one table + entry). colitur's own residual tie-break (alphabetical by slug, §6.1's + own uncited-convention record) picks "maurice..." over "thomas..." purely + because 'm' < 't' -- no rubrical warrant either way, so this is NOT + adjudicated colitur or missalemeum: verdict OPEN, the same explicit + permission the brief gives M11/M13 ("say so as an open item rather than + absorbing it"), register §6.1. Only 2027 falls in this window with this + EXACT collision -- 2026's 22 September is an ordinary (non-Ember) + Tuesday, so Thomas of Villanova simply wins the day outright on both + sides (colitur: observed; missalemeum: title) with Maurice commemorated + alongside him identically on both -- no tie to observe that year. *) +let m17_dates = [ "2027-09-22" ] + let layer_m_reason (c : colitur_row) (o : oracle_row) diffs = if diffs = [] then None else if List.mem c.c_date m1_dates && subset diffs [ Rank; Colour_f; Comm_presence ] then Some "M1" @@ -506,7 +695,16 @@ let layer_m_reason (c : colitur_row) (o : oracle_row) diffs = else if List.mem c.c_date m10_dates && diffs = [ Comm_presence ] then Some "M10" else if List.mem c.c_date m11_dates && diffs = [ Comm_presence ] then Some "M11" else if List.mem c.c_date m12_dates && diffs = [ Comm_count ] then Some "M12" - else if List.mem c.c_date m13_dates && subset diffs [ Rank; Colour_f ] then Some "M13" + (* M13's own subset widened (Task B): the same underlying gap (the Seven + Sorrows commemoration is never constructed at all) now ALSO surfaces + as [Comm_identity_unresolved] on this date, since colitur's own + admitted commemoration there is temporal-origin (no English name) -- + one root cause, one id, both axes it touches. *) + else if List.mem c.c_date m13_dates && subset diffs [ Rank; Colour_f; Comm_identity_unresolved ] then + Some "M13" + else if List.mem c.c_date m15_dates && diffs = [ Comm_identity_unresolved ] then Some "M15" + else if List.mem c.c_date m16_dates && diffs = [ Comm_identity_mismatch ] then Some "M16" + else if List.mem c.c_date m17_dates && diffs = [ Comm_identity_mismatch ] then Some "M17" else None (* ---------------------------------------------------------------------- *) @@ -550,17 +748,24 @@ let classify oracle colitur = | None -> (c, o, Unexplained diffs)) oracle colitur +let status_to_string = function Cel.Feast -> "Feast" | Cel.Commemoration_only -> "Commemoration_only" + +let describe_comm (slug, rank, status, name) = + Printf.sprintf "%s(rank=%d,status=%s,name=%s)" slug rank (status_to_string status) + (match name with Some n -> Printf.sprintf "%S" n | None -> "NONE") + let describe_unexplained (c : colitur_row) (o : oracle_row) diffs = Printf.sprintf "%s: %s differ -- colitur=(rank=%d colour=%c comms=[%s]) oracle=(rank=%d colours=[%s] \ - title=%S comms=[%s] displaced=[%s])" + title=%S comms=[%s] comm_ids=[%s] displaced=[%s])" c.c_date (String.concat "," (List.map field_name diffs)) c.c_rank c.c_colour - (String.concat ";" c.c_commemorations) + (String.concat ";" (List.map describe_comm c.c_commemorations)) o.o_rank (String.concat "" (List.map (String.make 1) o.o_colours)) o.o_title (String.concat ";" o.o_commemorations) + (String.concat ";" o.o_commemoration_ids) (String.concat ";" o.o_displaced) let test_fixture_checksum () = @@ -578,9 +783,11 @@ let test_dates_align () = in Alcotest.(check (list (pair string string))) "no misaligned dates" [] mismatched -(* The core assertion: every one of the 42 raw differences across all four +(* The core assertion: every one of the 63 raw differences across all six axes (rank, colour-membership, commemoration-presence, commemoration- - count) is named in the cited allow-list. Nothing else passes silently. *) + count, and, since Task B/ef-rg16a, commemoration-identity-mismatch and + commemoration-identity-unresolved) is named in the cited allow-list. + Nothing else passes silently. *) let test_no_unexplained_differences () = let oracle, colitur = compare_streams () in let classified = classify oracle colitur in @@ -616,6 +823,115 @@ let test_layer_m_counts_match_citations () = undeclared" expected actual +(* Corroboration (Task B): for every day where the identity axis is CLEAN + (no [Comm_identity_mismatch]/[Comm_identity_unresolved] -- titles matched + as a set), independently cross-check missalemeum's own id-embedded rank + digit against colitur's OWN candidate rank for that SAME title -- a + SECOND, independent signal beyond string equality (this file's own + header: "used only as a corroborating check, never as the primary + signal"). Restricted to "sancti:"-origin ids (a TEMPORAL-origin oracle + commemoration has no colitur-side name to pair it with in the first + place, see [Comm_identity_unresolved]'s own days, which never reach here + since their identity axis is not clean). + + SPLIT BY STATUS -- a genuine finding from running this check, not the + originally-planned single assertion: EVERY ONE of colitur's 114 + [Commemoration_only] sanctoral entries carries a bootstrap-assigned + [rank Class3] (verified: `grep -c "status Commemoration_only"` and + `grep -B3 ... | grep -oP "rank Class\d" | sort | uniq -c` both give + 114/114 Class3, data/ef/sanctoral.sexp) -- but missalemeum's OWN id-rank + for a [Commemoration_only] match is NOT itself uniform: of the 175 + examined, 17 genuinely AGREE (oracle_rank = colitur_rank = 3, real + corroboration) and 158 show oracle_rank 4 against colitur's bootstrapped + 3 -- CORRECTED after the first version of this comment overclaimed "id + consistently encodes 4 for every entry" from the 158-strong failure list + alone, before the 17 agreeing rows (then hidden inside a too-strict + assertion) were found. Two acceptable shapes, then, not one: either the + ranks genuinely agree, or the specific oracle=4/colitur=3 gap holds. This + is not evidence of a title-matching error, and not new: it independently + corroborates the register's own already-open item (§6, "Commemoration- + only entries' inferred STATUS, not only rank" -- Task 16's own data + audit already flagged these ranks as bootstrap inferences, never + individually Missal-verified). Auditing which of the 114 are genuinely + Class3 and which were silently defaulted to it is that item's own scope, + not this task's -- so rather than either asserting a false universal + equality (which fails on the 158 KNOWN, tracked, out-of-scope rows) or + silently dropping the check, this pins the discovered SHAPE itself: + every [Commemoration_only] match is EITHER a real agreement OR the known + gap, with no THIRD pattern -- a real, falsifiable claim (any other + combination would break it) that stays honest about what remains + unverified. + + [Feast]-status candidates get the ORIGINAL, unrestricted check (any + disagreement at all is unexpected) -- but this window's own data never + actually offers a [Feast]-status LOSING candidate whose identity is + independently clean (checked: 0 in the 2026-2027 fixture -- a [Feast] + candidate here either wins its own day outright, in which case it is + never a commemoration at all, or the one day where it does lose, + 22 September 2027/Thomas of Villanova, is ITSELF the M17 tie-break + mismatch and so is excluded by the [identity_diff = None] guard before + ever reaching this check). Kept anyway, not deleted: a real disagreement + would still be reported the moment one becomes reachable (a wider oracle + window, or a data change), and this project's own vacuity catalogue + flags "an assertion true by construction" as a defect shape to avoid, + not "an assertion whose population happens to be empty in the one + fixture available" -- the CHECK still does real work when its input is + non-empty; it is the DATA, not the code, that is currently silent here. *) +let test_identity_rank_corroboration () = + let oracle, colitur = compare_streams () in + let feast_mismatches = ref [] in + let commemoration_only_checked = ref 0 in + let commemoration_only_surprises = ref [] in + List.iter2 + (fun (o : oracle_row) (c : colitur_row) -> + if + identity_diff c o = None && c.c_commemorations <> [] && o.o_commemorations <> [] + && List.length c.c_commemorations = List.length o.o_commemorations + then + List.iter2 + (fun title id -> + if is_sancti_id id then + match id_rank id with + | None -> () + | Some oracle_rank -> ( + match List.find_opt (fun (_, _, _, n) -> n = Some title) c.c_commemorations with + | Some (slug, colitur_rank, Cel.Feast, _) -> + if colitur_rank <> oracle_rank then + feast_mismatches := + Printf.sprintf "%s: %s oracle-id-rank=%d colitur-rank=%d" c.c_date slug + oracle_rank colitur_rank + :: !feast_mismatches + | Some (slug, colitur_rank, Cel.Commemoration_only, _) -> + incr commemoration_only_checked; + (* Two ACCEPTABLE shapes, not one -- corrected after first running this + check: most Commemoration_only entries genuinely agree + (oracle_rank = colitur_rank, real corroboration, 17 of the + population checked), and the rest show the KNOWN + oracle=4/colitur=3 convention gap (this comment's own header). + Anything OUTSIDE those two shapes (e.g. oracle=2 or a THIRD + colitur rank paired with a non-4 oracle rank) is a genuine + surprise. *) + if not (oracle_rank = colitur_rank || (oracle_rank = 4 && colitur_rank = 3)) then + commemoration_only_surprises := + Printf.sprintf "%s: %s oracle-id-rank=%d colitur-rank=%d (neither agrees nor \ + fits the known oracle=4/colitur=3 pattern)" + c.c_date slug oracle_rank colitur_rank + :: !commemoration_only_surprises + | None -> ())) + o.o_commemorations o.o_commemoration_ids) + oracle colitur; + Alcotest.(check (list string)) + "Feast-status matches: oracle id-rank agrees with colitur's own rank (none reachable in this window, \ + see this test's own comment; the check still runs)" + [] (List.rev !feast_mismatches); + Alcotest.(check (list string)) + "Commemoration_only-status matches: every one fits the KNOWN oracle=4/colitur=3 convention gap -- any \ + other combination would be a genuine, new surprise" + [] (List.rev !commemoration_only_surprises); + (* Vacuity guard for the branch that IS populated in this window. *) + Alcotest.(check bool) "the Commemoration_only population actually examined is non-trivial" true + (!commemoration_only_checked > 50) + let suite = ( "oracle (missalemeum, EF, 2026-2027)", [ Alcotest.test_case "fixture SHA-256 matches its provenance note" `Quick test_fixture_checksum; @@ -623,5 +939,8 @@ let suite = Alcotest.test_case "every difference is named in the cited allow-list -- none unexplained" `Quick test_no_unexplained_differences; Alcotest.test_case "allow-list counts match data/ef/expected-divergences-missalemeum.sexp exactly" - `Quick test_layer_m_counts_match_citations + `Quick test_layer_m_counts_match_citations; + Alcotest.test_case + "identity corroboration: matched titles' oracle id-rank agrees with colitur's own rank" `Quick + test_identity_rank_corroboration ] ) |
