From fcfce4ba0ab4ea93837b647deaa5292300aaad5e Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 12 Aug 2026 03:18:15 +0200 Subject: test: differential vs lectio 2005-2050 with a cited allow-list Validation layer 3 (design spec's five): compares colitur's real day-by-day EF output against lectio (sibling project, Go), 2005-2050, one line per civil day. Of 16801 day-pairs, 11206 already agree on the seven leading columns; the 5595 that don't resolve into exactly 25 distinct field-diff signatures, all triaged. Three strictly separate layers, per the controller's ruling (the brief's single flat allow-list assumed a handful of differences, not 5595): - Layer A (test_differential.ml, norm_season/norm_slug): vocabulary. An explicit, closed table of naming synonyms with no liturgical substance (lectio's easter/christmas vs colitur's paschaltide/christmastide; a handful of slugs that are two names for the identical office). No wildcards -- every entry is a literal string pair. - Layer B (strip_epiphany_index): numbering. The one slug family whose lectio/colitur index offset is not a constant (Time-after-Epiphany week numbering, register 3c#5) has its embedded digit stripped to a common form on both sides before comparing; rank and colour stay fully compared. - Layer C (data/ef/expected-divergences.sexp): the cited allow-list. Ten genuine liturgical disagreements, each citing its RG paragraph and naming which engine is right (always colitur, verified against the Missal/register, never against lectio's own behaviour). This is the only layer permitted to cover a difference in rank, colour, or which celebration is observed. Five extend or restate register 3c's already- documented divergences (season boundary, Sunday I-class, Advent Ember ferias, Rogations); five are new, found and adjudicated in this task (Lent Ember days, the Nativity Octave, Ember-day-vs-saint precedence, the St Joseph transfer off a Lent Sunday, and the 2011 Sacred Heart / Precious Blood / Visitation collision). expected_rows on each entry is an exact regression pin, asserted by the test, not documentation. 13 January (register 6's long-open "Baptism of the Lord" item) is confirmed empirically fixed already -- Task 11's sanctoral wiring closed it before this task started -- so it is not allow-listed; the only residual difference there is the season boundary already covered by C1. Two stated limits carried from the brief (commemorations are not comparable; lectio's own EF oracle asserts season only, 2025-2026 only, so a rank/colour difference is not presumptive evidence against colitur) plus a third found during this task (the week column is a display convention on both sides, not a liturgical fact, and is not compared at all) are documented in the test file's own doc comment. Fixture: test/fixtures/lectio-ef-2005-2050.txt, committed as plain text (1.4 MB), generated by lectio commit 2386a45; provenance recorded in the sibling .provenance file. Colitur's side is recomputed fresh from the library on every run, through the same Calendar/Rite_ef pipeline `colitur day` uses, not the compiled binary. Proved the harness has teeth by two reverted perturbations: a genuine colour difference injected into a fully-covered fixture row fails the "no unexplained differences" check with the exact mismatched row printed; a one-row drift in an allow-list entry's expected_rows fails the count check independently, showing it is not merely a duplicate of the first assertion. 236/236 tests green, clean-build verified, deterministic across OCAMLRUNPARAM=R. --- test/test_differential.ml | 494 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 494 insertions(+) create mode 100644 test/test_differential.ml (limited to 'test/test_differential.ml') diff --git a/test/test_differential.ml b/test/test_differential.ml new file mode 100644 index 0000000..aea1dac --- /dev/null +++ b/test/test_differential.ml @@ -0,0 +1,494 @@ +(* Task 15: differential harness vs lectio (sibling project, Go), EF only, + 2005-2050 -- validation layer 3 of the design spec's five (colitur CLAUDE.md + "Validation" section; layers 1-2, Types and Property, are already built and + green). lectio's own EF stream is committed as a fixture + (test/fixtures/lectio-ef-2005-2050.txt, provenance in the sibling + .provenance file next to it); colitur's side is recomputed fresh from this + library on every run, through the SAME pipeline `colitur day` uses + (Colitur_kernel.Calendar over the real data/ef/sanctoral.sexp + + adjustments.sexp), not the compiled binary. + + *** THREE STATED LIMITS THIS COMPARATOR DOES NOT PRETEND TO EXCEED *** + + 1. Commemorations are NOT compared. lectio's trailing "+other-slug" tokens + are the LOSING sanctoral candidates for the day (its own dumper's doc + comment says so), not an RG 111 admitted set -- lectio has no RG 111 + admission logic at all. Comparing that column would compare colitur's + real admitted commemorations against lectio's rejects, which proves + nothing. Only the seven leading columns (date weekday season week slug + rank colour) are read from either stream. + + 2. lectio's own EF oracle test (~/git/projects/lectio, + internal/calendar/oracle_ef_test.go) asserts ONLY season, strictly, + against missalemeum, and ONLY for 2025-2026. Rank and colour are logged + there, never asserted. So lectio's advertised "0-error vs references + 2005-2050" covers season-exactness for the EF, not full-row-exactness. + A rank or colour difference below is therefore NOT presumptive evidence + that colitur is wrong -- every one is adjudicated against the Missal/RG + register directly (data/ef/expected-divergences.sexp), never against + lectio's say-so. (Layer 4, the oracle differential against missalemeum + itself, is what actually validates rank and colour; that is a later + task.) + + 3. The WEEK column is not compared, at all, on either side. Two reasons, + both confirmed by reading lectio's own source + (cmd/lectio-ef-dump/main.go's doc comment): first, lectio prints week + "0" (rendered "-") "where no season week applies, e.g. named I class + feasts and per annum green-season ferias" -- an internal DISPLAY + convention of lectio's, not a liturgical fact, so there is nothing to + cross-check it against. Second, even where both sides print a real + number, the two engines anchor Time-after-Epiphany week numbering + differently (colitur: the first Sunday on/after Epiphany; lectio: a + fixed offset from 6 January -- register §3c item 5) and the offset + between them is NOT constant (it depends on which weekday 6 January + falls on, and re-synchronises mid-season), so no clean formula-based + check is possible without reimplementing lectio's own algorithm here. + Slug identity, rank and colour -- where the real liturgical substance + lives -- remain fully and separately compared; only the bare integer + is out of scope. See the report for the concrete rows this weakens. + + *** THE THREE-LAYER DESIGN (controller ruling, Task 15 dispatch) *** + + Of 16801 day-pairs (2005-2050), 11206 already match on the seven columns. + Of the 5595 that don't, exactly 25 distinct (field-diff) signatures cover + all of them (task-15-class-summary.txt). They resolve into three strictly + separate layers: + + - Layer A (this file's [norm_season]/[norm_slug]): vocabulary. A + declarative, explicit, closed table of naming synonyms that carry no + liturgical substance -- lectio's "easter"/"christmas" ARE colitur's + "paschaltide"/"christmastide"; a handful of slugs are two different + engines' names for the identical office (Christmas Vigil, Holy Name + Sunday, the Pentecost-octave Ember days, the two Passiontide weeks). + Every entry is a literal string pair, never a pattern -- widening this + to a wildcard is exactly how a real bug would get hidden, so it is not + done even where it would shorten the table. + + - Layer B ([strip_epiphany_index]): numbering. The ONE case in the whole + 5595 where a slug's embedded index genuinely cannot be reconciled by a + literal table (Time-after-Epiphany's non-constant offset, limit 3 + above) -- both sides' embedded week digit is stripped to a common + family+weekday form before comparing, while rank and colour (which + carry no week-index information for an ordinary green-season + feria/Sunday) remain fully compared, so a genuine identity bug in this + family would still be caught by everything except the digit itself. + + - Layer C (data/ef/expected-divergences.sexp, matched by + [layer_c_reason] below): the CITED allow-list. Ten genuine liturgical + disagreements, each citing its RG paragraph and stating which engine is + right (always colitur, verified against the Missal/register, never + against lectio's own behaviour -- "lectio does it differently" is not + itself a justification anywhere in this file). This is the ONLY layer + that may cover a difference in rank, colour, or which celebration is + observed; A and B never do (enforced structurally below: A/B only ever + touch the season/slug fields, and Layer C's predicates each require an + exact, narrow field-diff SET, not "anything goes"). + + 13 January (register §6's long-open "Baptism of the Lord" item) is + EMPIRICALLY CONFIRMED FIXED, not allow-listed: colitur's slug/rank/colour + for 13 January already equal lectio's exactly, in all 46 years (the + sanctoral wiring landed in Task 11's "13 Jan now resolves to the Baptism + of the Lord" review note). The only residual difference there is season + (covered by Layer C's C1, the Jan 6-13 boundary) -- see the report. *) + +module Cal = Colitur_kernel.Calendar +module Layer = Colitur_kernel.Layer +module Overlay = Colitur_kernel.Overlay +module LD = Colitur_kernel.Liturgical_day +module Slug = Colitur_kernel.Slug +module Date = Colitur_kernel.Date +module Cel = Colitur_kernel.Celebration +module Colour = Colitur_kernel.Colour +module V = Rite_ef.Vocab_ef + +(* Same relative paths test_rite_ef.ml/test_sanctoral_ef.ml use: dune test + runs from _build/default/test/. *) +let sanctoral_path = "../data/ef/sanctoral.sexp" +let adjustments_path = "../data/ef/adjustments.sexp" +let fixture_path = "fixtures/lectio-ef-2005-2050.txt" +let allow_list_path = "../data/ef/expected-divergences.sexp" + +let real_layer () = + let layer = + match Layer.load V.rank_of_sexp sanctoral_path with + | Ok l -> l + | Error e -> Alcotest.failf "%s: failed to load: %s" sanctoral_path e + in + let overlay = + match Overlay.load V.rank_of_sexp adjustments_path with + | Ok o -> o + | Error e -> Alcotest.failf "%s: failed to load: %s" adjustments_path e + in + let layer, diagnostics = Overlay.apply layer overlay in + Alcotest.(check (list string)) "the committed overlay applies cleanly, no diagnostics" [] + (List.map Overlay.diagnostic_to_string diagnostics); + layer + +(* --- The seven-column row both streams share (commemorations excluded, --- *) +(* limit 1 above). *) +type row = { + date : string; + weekday : string; + season : string; + week : string; + slug : string; + rank : string; + colour : string; +} + +let read_lines path = + let ic = open_in path in + let rec loop acc = + match input_line ic with + | line -> loop (line :: acc) + | exception End_of_file -> + close_in ic; + List.rev acc + in + loop [] + +let row_of_line line = + match String.split_on_char ' ' line with + | date :: weekday :: season :: week :: slug :: rank :: colour :: _others -> + { date; weekday; season; week; slug; rank; colour } + | _ -> Alcotest.failf "malformed fixture line (fewer than 7 fields): %S" line + +let lectio_rows () = List.map row_of_line (read_lines fixture_path) + +(* Recomputes colitur's day-by-day output straight from the library -- + exactly [Colitur_kernel.Calendar.year] + [Rite_ef.context] over the real + committed data, the same pipeline bin/main.ml's `colitur day` runs (that + file's own [day_report]/[day_line] comments explain the two-liturgical- + year-per-civil-year indexing this mirrors). Duplicated here rather than + shared with bin/main.ml (an executable, not a library) -- the same choice + test_rite_ef.ml already made for [real_layer] above. *) +let colitur_rows_2005_2050 () = + let layer = real_layer () in + let by_rata : (int, (V.season, V.rank) LD.t) Hashtbl.t = Hashtbl.create 20000 in + for y = 2004 to 2050 do + let days = Cal.year Rite_ef.context layer y in + Array.iter (fun (d : (V.season, V.rank) LD.t) -> Hashtbl.replace by_rata (Date.to_rata d.LD.date) d) days + done; + let mk y m d = match Date.make ~year:y ~month:m ~day:d with Ok t -> t | Error e -> failwith e in + let rows = ref [] in + for y = 2005 to 2050 do + let d = ref (mk y 1 1) in + let stop = mk y 12 31 in + while Date.compare !d stop <= 0 do + (match Hashtbl.find_opt by_rata (Date.to_rata !d) with + | Some day -> + let t = day.LD.temporal in + let cel = day.LD.observed in + let week = + match t.Colitur_kernel.Temporal.week with Some n -> string_of_int n | None -> "-" + in + rows := + { date = Date.to_iso8601 day.LD.date; + weekday = Date.weekday_to_string t.Colitur_kernel.Temporal.weekday; + season = V.season_to_string t.Colitur_kernel.Temporal.season; + week; + slug = Slug.to_string cel.Cel.slug; + rank = V.rank_to_string cel.Cel.rank; + colour = Colour.to_string cel.Cel.colour + } + :: !rows + | None -> Alcotest.failf "internal error: no resolved day for %s" (Date.to_iso8601 !d)); + d := Date.add_days !d 1 + done + done; + List.rev !rows + +(* ---------------------------------------------------------------------- *) +(* Layer A: vocabulary. Explicit, closed tables; no wildcards, no pattern *) +(* matching against arbitrary content -- every case below is a literal *) +(* string compared against a literal string. *) +(* ---------------------------------------------------------------------- *) + +(* lectio's season word -> colitur's season word. The ONLY two seasons that + are spelled differently; every other season word is shared verbatim + (both call Advent "advent", Lent "lent", etc.). *) +let norm_season = function + | "easter" -> "paschaltide" + | "christmas" -> "christmastide" + | s -> s + +let weekdays = [ "monday"; "tuesday"; "wednesday"; "thursday"; "friday"; "saturday" ] + +(* lectio's slug -> the colitur slug for the SAME office, where the two + projects simply chose different names. [month]/[lectio_rank] disambiguate + the two cases where lectio reuses one slug for what colitur names two + different ways (see each branch's own comment). *) +let rec norm_slug ~month ~lectio_rank slug = + if String.equal slug "vigil-of-christmas" then "ef-nativity-vigil" + (* register §6: lectio names its vigils "vigil-of-X" (prefix); colitur's + own convention is "X-vigil" (suffix). Same celebration -- confirmed a + genuine duplicate in Task 11, where colitur's overlay suppresses + lectio's key entirely from the sanctoral layer. *) + else if + (* "ef-christmas-sunday-0"/"ef-christmas-0-" mean TWO different + things in lectio depending on which window they fall in: the Sunday + within Holy Name week (2-5 January, colitur's own named feast) or an + ordinary day within the Nativity Octave (26-31 December, where colitur + uses its own ef-nativity-octave-day-N naming instead, Layer C's C6 -- + that window keeps a genuine RANK difference too, so it must not be + absorbed here). Gating on [month = 1] keeps this table from ever + touching the December occurrence. *) + month = 1 + then + if String.equal slug "ef-christmas-sunday-0" then "ef-holy-name-sunday" + else + match List.find_opt (fun wd -> String.equal slug ("ef-christmas-0-" ^ wd)) weekdays with + | Some wd -> "ef-christmas-1-" ^ wd + | None -> norm_slug_rest ~lectio_rank slug + else norm_slug_rest ~lectio_rank slug + +and norm_slug_rest ~lectio_rank slug = + (* Passiontide: lectio never distinguishes Passion week (colitur's week 1, + class-3 ferias) from Holy week (colitur's week 2, class-1 ferias) in the + slug -- both print "ef-passiontide-0-". Rank, independently and + strictly compared elsewhere, is what actually tells the two weeks apart + (RG 91 entries 22 vs 2/7), so using it here to pick the expected colitur + digit does not launder away a genuine identity bug: a colitur bug that + mixed up the two weeks would, on the evidence available in this stream, + also very likely show up as a rank mismatch of its own. *) + match List.find_opt (fun wd -> String.equal slug ("ef-passiontide-0-" ^ wd)) weekdays with + | Some wd -> ( + match lectio_rank with + | "class-3" -> "ef-passiontide-1-" ^ wd + | "class-1" -> "ef-passiontide-2-" ^ wd + | _ -> slug) + | None -> ( + match slug with + | "ef-easter-8-wednesday" -> "ef-pentecost-ember-wed" + | "ef-easter-8-friday" -> "ef-pentecost-ember-fri" + | "ef-easter-8-saturday" -> "ef-pentecost-ember-sat" + | s -> s) + +(* ---------------------------------------------------------------------- *) +(* Layer B: numbering (register §3c item 5). The Time-after-Epiphany *) +(* week-index embedded in a slug is stripped to a common form on BOTH *) +(* sides before comparing -- see limit 3 in this file's header comment *) +(* for why a table (Layer A's tool) cannot do this instead. *) +(* ---------------------------------------------------------------------- *) + +let starts_with ~prefix s = + let lp = String.length prefix in + String.length s >= lp && String.equal (String.sub s 0 lp) prefix + +let is_digit_string s = s <> "" && String.for_all (fun c -> c >= '0' && c <= '9') s + +let strip_epiphany_index slug = + let prefix = "ef-time-after-epiphany-" in + if not (starts_with ~prefix slug) then slug + else + let rest = String.sub slug (String.length prefix) (String.length slug - String.length prefix) in + match String.index_opt rest '-' with + | None -> slug + | Some i -> + let left = String.sub rest 0 i in + let right = String.sub rest (i + 1) (String.length rest - i - 1) in + if is_digit_string left && List.mem right weekdays then prefix ^ right + else if String.equal left "sunday" && is_digit_string right then prefix ^ "sunday" + else slug + +(* ---------------------------------------------------------------------- *) +(* Field-diff computation: applies Layers A and B, then reports exactly *) +(* which of the SIX substantive columns still differ (week is never *) +(* inspected at all -- limit 3). weekday is included defensively: dates *) +(* are checked 1:1 aligned before this runs, so it should never fire, and *) +(* if it ever does that is real signal, not noise to normalise away. *) +(* ---------------------------------------------------------------------- *) + +type field = Weekday | Season | Slug_f | Rank | Colour_f + +let field_name = function + | Weekday -> "weekday" + | Season -> "season" + | Slug_f -> "slug" + | Rank -> "rank" + | Colour_f -> "colour" + +let month_of_date date = int_of_string (String.sub date 5 2) + +let diff_fields (l : row) (c : row) = + let m = month_of_date l.date in + let l_season = norm_season l.season in + let l_slug = strip_epiphany_index (norm_slug ~month:m ~lectio_rank:l.rank l.slug) in + let c_slug = strip_epiphany_index c.slug in + List.filter_map + (fun x -> x) + [ (if String.equal l.weekday c.weekday then None else Some Weekday); + (if String.equal l_season c.season then None else Some Season); + (if String.equal l_slug c_slug then None else Some Slug_f); + (if String.equal l.rank c.rank then None else Some Rank); + (if String.equal l.colour c.colour then None else Some Colour_f) + ] + +(* ---------------------------------------------------------------------- *) +(* Layer C: the cited allow-list (data/ef/expected-divergences.sexp). *) +(* Each predicate below names the [id] it matches; the sexp file carries *) +(* that id's citation, verdict and expected row count. A predicate fires *) +(* only on an EXACT, narrow field-diff set -- never "any difference at *) +(* all" -- so it cannot silently absorb a difference outside what its own *) +(* citation actually explains. *) +(* ---------------------------------------------------------------------- *) + +let subset xs ys = List.for_all (fun x -> List.mem x ys) xs +let day_of_date date = int_of_string (String.sub date 8 2) + +let advent_feria_slug slug = + List.exists + (fun wk -> List.exists (fun wd -> String.equal slug (Printf.sprintf "ef-advent-%d-%s" wk wd)) weekdays) + [ 3; 4 ] + +let sunday_iclass_slugs = + [ "ef-advent-sunday-2"; "ef-advent-sunday-4"; "ef-lent-sunday-1"; "ef-lent-sunday-2"; "ef-lent-sunday-3" ] + +let rose_sunday_slugs = [ "ef-advent-sunday-3"; "ef-lent-sunday-4" ] + +(* [layer_c_reason l c diffs] returns the [data/ef/expected-divergences.sexp] + [id] this row-pair's remaining (post Layer A/B) diff set belongs to, or + [None] if nothing here explains it (a genuine, uncovered failure). *) +let layer_c_reason (l : row) (c : row) diffs = + let m = month_of_date l.date and d = day_of_date l.date in + if diffs = [] then None + else if m = 1 && d >= 6 && d <= 13 && subset diffs [ Season; Colour_f; Slug_f ] then Some "C1" + else if List.mem c.slug sunday_iclass_slugs && diffs = [ Rank ] then Some "C2" + else if List.mem c.slug rose_sunday_slugs && subset diffs [ Rank; Colour_f ] then Some "C3" + else if advent_feria_slug c.slug && diffs = [ Rank ] then Some "C4" + else if starts_with ~prefix:"ef-lent-ember-" c.slug && subset diffs [ Slug_f; Rank ] then Some "C5" + else if m = 12 && (d = 29 || d = 30 || d = 31) && subset diffs [ Slug_f; Rank ] then Some "C6" + else if (String.equal c.slug "matthew" || String.equal c.slug "thomas") && subset diffs [ Slug_f; Colour_f ] + then Some "C7" + else if + (String.equal c.slug "ef-rogation-monday" || String.equal c.slug "ef-rogation-tuesday") + && subset diffs [ Season; Slug_f; Colour_f ] + then Some "C8" + else if + (String.equal l.slug "joseph-spouse-of-the-bl-virgin-mary" + || String.equal c.slug "joseph-spouse-of-the-bl-virgin-mary") + && subset diffs [ Season; Slug_f; Rank; Colour_f ] + then Some "C9" + else if (String.equal l.date "2011-07-02" || String.equal l.date "2011-07-04") then Some "C10" + else None + +(* ---------------------------------------------------------------------- *) +(* data/ef/expected-divergences.sexp loading -- a plain sequence of *) +(* top-level records (not one wrapping list: see the file's own header). *) +(* ---------------------------------------------------------------------- *) + +open Sexplib0.Sexp_conv + +type allow_entry = { id : string; citation : string; verdict : string; note : string; expected_rows : int } +[@@deriving sexp] + +let load_allow_list () = + let sexps = + try Sexplib.Sexp.load_sexps allow_list_path + with e -> Alcotest.failf "%s: failed to load: %s" allow_list_path (Printexc.to_string e) + in + List.map allow_entry_of_sexp sexps + +(* ---------------------------------------------------------------------- *) +(* The comparison itself, run once and shared by every test case below *) +(* (Alcotest test cases are independent processes-in-a-list, not free to *) +(* share mutable state across a suite, so this recomputes per call -- *) +(* acceptable: colitur's own 1583-9999 property sweep runs orders of *) +(* magnitude more days in tens of seconds, and this is 16801 civil days, *) +(* once per test case, twice total). *) +(* ---------------------------------------------------------------------- *) + +type outcome = Matched | Explained of string | Unexplained of field list + +let compare_streams () = + let lectio = lectio_rows () in + let colitur = colitur_rows_2005_2050 () in + (lectio, colitur) + +let classify lectio colitur = + List.map2 + (fun (l : row) (c : row) -> + if not (String.equal l.date c.date) then + Alcotest.failf "streams misaligned: lectio %s vs colitur %s" l.date c.date; + let diffs = diff_fields l c in + if diffs = [] then (l, c, Matched) + else + match layer_c_reason l c diffs with + | Some id -> (l, c, Explained id) + | None -> (l, c, Unexplained diffs)) + lectio colitur + +let describe_unexplained (l : row) (c : row) diffs = + Printf.sprintf "%s: %s differ -- lectio=(%s %s %s %s %s) colitur=(%s %s %s %s %s)" l.date + (String.concat "," (List.map field_name diffs)) + l.weekday l.season l.slug l.rank l.colour c.weekday c.season c.slug c.rank c.colour + +(* Dates align 1:1 in the same order on both streams (both are one line per + civil day, 2005-01-01..2050-12-31 -- see the fixture's own provenance + note and [colitur_rows_2005_2050]'s construction). A silent misalignment + would make every subsequent comparison meaningless -- checked first, on + its own, rather than trusted. *) +let test_dates_align () = + let lectio, colitur = compare_streams () in + Alcotest.(check int) "both streams have 16801 rows (46*365 + 11 leap days)" 16801 (List.length lectio); + Alcotest.(check int) "colitur recomputed the same number of rows" (List.length lectio) (List.length colitur); + let mismatched = + List.filter_map + (fun (l, c) -> if String.equal l.date c.date then None else Some (l.date, c.date)) + (List.combine lectio colitur) + in + Alcotest.(check (list (pair string string))) "no misaligned dates" [] mismatched + +(* The core assertion: every one of the 5595 raw differences is either + normalised away (Layers A/B) or named in the cited allow-list (Layer C). + Nothing else is permitted to pass silently. *) +let test_no_unexplained_differences () = + let lectio, colitur = compare_streams () in + let classified = classify lectio colitur in + let unexplained = + List.filter_map + (fun (l, c, outcome) -> + match outcome with Unexplained diffs -> Some (describe_unexplained l c diffs) | _ -> None) + classified + in + Alcotest.(check (list string)) "no differences outside Layers A/B/C" [] unexplained + +(* Teeth, not just green: EVERY Layer C entry's actual row count over this + fixture must equal what data/ef/expected-divergences.sexp declares, in + BOTH directions -- an id used by [layer_c_reason] that is missing from + the sexp file, an id declared but never matched, or a count that has + drifted either up or down, all fail loudly. A silent drift here is + exactly the "allow-list absorbs a new bug" failure mode this task was + warned about. *) +let test_layer_c_counts_match_citations () = + let lectio, colitur = compare_streams () in + let classified = classify lectio colitur in + let actual_counts = Hashtbl.create 16 in + List.iter + (fun (_, _, outcome) -> + match outcome with + | Explained id -> + Hashtbl.replace actual_counts id (1 + Option.value ~default:0 (Hashtbl.find_opt actual_counts id)) + | _ -> ()) + classified; + let declared = load_allow_list () in + let expected = + List.sort compare (List.map (fun e -> (e.id, e.expected_rows)) declared) + in + let actual = + List.sort compare + (Hashtbl.fold (fun id n acc -> (id, n) :: acc) actual_counts []) + in + Alcotest.(check (list (pair string int))) + "every allow-list id's actual row count matches its citation's expected_rows, and no id is unused or \ + undeclared" + expected actual + +let suite = + ( "differential (lectio, EF, 2005-2050)", + [ Alcotest.test_case "streams are 16801 rows each, dates aligned 1:1" `Quick test_dates_align; + Alcotest.test_case "every difference is normalised (A/B) or cited (C) -- none unexplained" `Quick + test_no_unexplained_differences; + Alcotest.test_case "Layer C counts match data/ef/expected-divergences.sexp exactly" `Quick + test_layer_c_counts_match_citations + ] ) -- cgit v1.3 From efc1b303417388c75f27403952b506dc4d4e03d7 Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 12 Aug 2026 03:37:37 +0200 Subject: test(differential): gate C1/C6 on slug family, pin the fixture's SHA-256 Fix round 1 on Task 15's review, four findings closed. C1 and C6 previously gated on calendar date alone -- unlike every other Layer C entry, which also checks the actual colitur slug that won. A future sanctoral change producing an unrelated celebration with the same diff shape (e.g. a promoted 29-31 December saint landing on C6's own [Slug_f; Rank]) would have been silently absorbed under a citation that has nothing to do with the real cause. Both entries now require, whenever Slug_f is in the diff set, that colitur's own slug be a member of an explicit closed list (jan_6_13_slug, nativity_octave_day_slugs) -- the same style advent_feria_slug already used for C4. Constructed the exact substitution via a temporary (unshipped) overlay Replace directive promoting thomas-becket to Class1 Feast White; confirmed the tightened predicate refuses to absorb it (46 unexplained rows, one per year); reverted. C10 gained the diff-field-subset restriction every other entry already had (it previously fired on the two pinned dates regardless of which fields differed). Fixed a stale doc comment ("SIX" columns; field has five constructors). The fixture's provenance note now records its SHA-256, and the suite asserts it as its own first test case (test_fixture_checksum), so a hand-edited or partially re-copied fixture fails loudly instead of silently becoming an unlabelled snapshot. Computed via the system sha256sum binary through Sys.command plus a redirected-output temp file -- no new library dependency, not even unix. Constructed a one-byte fixture edit and confirmed the checksum test fails independently of the row-comparison test; reverted. 237/237 tests green, clean-build verified. --- test/fixtures/lectio-ef-2005-2050.provenance | 8 ++ test/test_differential.ml | 109 ++++++++++++++++++++++++--- 2 files changed, 107 insertions(+), 10 deletions(-) (limited to 'test/test_differential.ml') diff --git a/test/fixtures/lectio-ef-2005-2050.provenance b/test/fixtures/lectio-ef-2005-2050.provenance index e738484..872517b 100644 --- a/test/fixtures/lectio-ef-2005-2050.provenance +++ b/test/fixtures/lectio-ef-2005-2050.provenance @@ -5,6 +5,14 @@ Produced by the sibling project ~/git/projects/lectio at commit: 2386a45 cmd(lectio-ef-dump): new EF calendar dumper for colitur's differential oracle (branch polish-ui-and-calendar at the time; see that repo's own log) +SHA-256: 2ca3eeeda4e7a0406c4d004c1b2003fc0df671aca9af18a1b506543a721c8bac +(same digest as [fixture_sha256] in test/test_differential.ml, ASSERTED by +that suite's own "fixture SHA-256 matches its provenance note" test -- this +line documents the pin for a reader who never runs the suite; that test is +what actually enforces it. Regenerate both together, deliberately, after +re-running the exact command below -- never by copying a mismatch's actual +value back in, which would defeat the pin's purpose.) + Exact command: cd ~/git/projects/lectio && go run ./cmd/lectio-ef-dump 2005 2050 diff --git a/test/test_differential.ml b/test/test_differential.ml index aea1dac..dd601a1 100644 --- a/test/test_differential.ml +++ b/test/test_differential.ml @@ -2,11 +2,13 @@ 2005-2050 -- validation layer 3 of the design spec's five (colitur CLAUDE.md "Validation" section; layers 1-2, Types and Property, are already built and green). lectio's own EF stream is committed as a fixture - (test/fixtures/lectio-ef-2005-2050.txt, provenance in the sibling - .provenance file next to it); colitur's side is recomputed fresh from this - library on every run, through the SAME pipeline `colitur day` uses - (Colitur_kernel.Calendar over the real data/ef/sanctoral.sexp + - adjustments.sexp), not the compiled binary. + (test/fixtures/lectio-ef-2005-2050.txt, provenance -- including the + SHA-256 this file's own [test_fixture_checksum] asserts, so a hand-edit or + partial re-copy of the fixture fails loudly rather than silently becoming + an unlabelled snapshot -- in the sibling .provenance file next to it); + colitur's side is recomputed fresh from this library on every run, through + the SAME pipeline `colitur day` uses (Colitur_kernel.Calendar over the + real data/ef/sanctoral.sexp + adjustments.sexp), not the compiled binary. *** THREE STATED LIMITS THIS COMPARATOR DOES NOT PRETEND TO EXCEED *** @@ -108,6 +110,49 @@ let adjustments_path = "../data/ef/adjustments.sexp" let fixture_path = "fixtures/lectio-ef-2005-2050.txt" let allow_list_path = "../data/ef/expected-divergences.sexp" +(* The fixture's own provenance note (test/fixtures/lectio-ef-2005-2050.provenance) + records this same digest, so it is discoverable by a reader who never runs + the suite. Recorded here too, and ASSERTED (fix round 1, finding 4): a + provenance note is enough to REGENERATE the fixture but says nothing about + whether the committed bytes still match the commit they claim to come + from -- an unnoticed hand-edit or partial re-copy would silently turn the + whole oracle into an unlabelled snapshot of whatever someone last ran. + Regenerate this constant (and the provenance note's copy) together, + deliberately, after re-running the exact command the provenance note + names -- never by copying the actual value back in to make a mismatch + pass, which would defeat the point of pinning it at all. *) +let fixture_sha256 = "2ca3eeeda4e7a0406c4d004c1b2003fc0df671aca9af18a1b506543a721c8bac" + +(* Same technique tools/bootstrap_sanctoral.ml already uses for this exact + purpose (that file's own comment: shelling out to the system's + [sha256sum], not an OCaml crypto library -- Task 15's deps are frozen). + Unlike that tool, this avoids even the (already-permitted, per that + file's own comment, "already in the switch") [unix] library: [Sys.command] + plus a redirected-to-file capture needs nothing beyond the Stdlib every + dune executable already links. Depends on [sha256sum] being on PATH, + which every environment this suite has actually run in (Debian, per + CLAUDE.md) provides via coreutils; a machine without it fails this check + with a command-not-found exit code rather than silently skipping it. *) +let sha256_of_file path = + let tmp = Filename.temp_file "colitur_differential_sha256" ".txt" in + Fun.protect + ~finally:(fun () -> try Sys.remove tmp with Sys_error _ -> ()) + (fun () -> + let cmd = Printf.sprintf "sha256sum %s > %s" (Filename.quote path) (Filename.quote tmp) in + let rc = Sys.command cmd in + if rc <> 0 then Alcotest.failf "sha256sum exited %d for %s (is it on PATH?)" rc path; + let ic = open_in tmp in + let line = + try input_line ic + with End_of_file -> + close_in ic; + Alcotest.failf "sha256sum produced no output for %s" path + in + close_in ic; + match String.index_opt line ' ' with + | Some i -> String.sub line 0 i + | None -> Alcotest.failf "unexpected sha256sum output for %s: %S" path line) + let real_layer () = let layer = match Layer.load V.rank_of_sexp sanctoral_path with @@ -293,7 +338,7 @@ let strip_epiphany_index slug = (* ---------------------------------------------------------------------- *) (* Field-diff computation: applies Layers A and B, then reports exactly *) -(* which of the SIX substantive columns still differ (week is never *) +(* which of the FIVE substantive columns still differ (week is never *) (* inspected at all -- limit 3). weekday is included defensively: dates *) (* are checked 1:1 aligned before this runs, so it should never fire, and *) (* if it ever does that is real signal, not noise to normalise away. *) @@ -346,18 +391,48 @@ let sunday_iclass_slugs = let rose_sunday_slugs = [ "ef-advent-sunday-3"; "ef-lent-sunday-4" ] +(* Fix round 1, finding 1: C1 and C6 (below) originally gated on calendar + date alone, with no slug/slug-family check -- unlike every other entry + here. The reviewer constructed the failure this leaves open: if a future + sanctoral regeneration made some OTHER 29-31 December celebration win the + day (colliding coincidentally with C6's own [Slug_f; Rank] diff shape), + it would be silently absorbed under "RG 91 entry 17, Nativity Octave" -- + a citation that has nothing to do with the real cause. [subset diffs + [...]] alone was never enough; the SLUG that actually won must also be + the one each citation is about. Both lists below are exact literals (the + Nativity Octave's three colitur-only day slugs; the closed set of + offices that can legitimately observe C1's Jan 6-13 window), not + patterns -- a slug outside them fails through to [None] instead of being + absorbed. *) +let jan_6_13_slug slug = + String.equal slug "ef-epiphany" + || String.equal slug "commemoration-of-the-baptism-of-the-lord" + || List.exists (fun wd -> String.equal slug ("ef-christmas-2-" ^ wd)) weekdays + +let nativity_octave_day_slugs = + [ "ef-nativity-octave-day-5"; "ef-nativity-octave-day-6"; "ef-nativity-octave-day-7" ] + (* [layer_c_reason l c diffs] returns the [data/ef/expected-divergences.sexp] [id] this row-pair's remaining (post Layer A/B) diff set belongs to, or [None] if nothing here explains it (a genuine, uncovered failure). *) let layer_c_reason (l : row) (c : row) diffs = let m = month_of_date l.date and d = day_of_date l.date in if diffs = [] then None - else if m = 1 && d >= 6 && d <= 13 && subset diffs [ Season; Colour_f; Slug_f ] then Some "C1" + else if + m = 1 && d >= 6 && d <= 13 + && subset diffs [ Season; Colour_f; Slug_f ] + && (not (List.mem Slug_f diffs) || jan_6_13_slug c.slug) + then Some "C1" else if List.mem c.slug sunday_iclass_slugs && diffs = [ Rank ] then Some "C2" else if List.mem c.slug rose_sunday_slugs && subset diffs [ Rank; Colour_f ] then Some "C3" else if advent_feria_slug c.slug && diffs = [ Rank ] then Some "C4" else if starts_with ~prefix:"ef-lent-ember-" c.slug && subset diffs [ Slug_f; Rank ] then Some "C5" - else if m = 12 && (d = 29 || d = 30 || d = 31) && subset diffs [ Slug_f; Rank ] then Some "C6" + else if + m = 12 + && (d = 29 || d = 30 || d = 31) + && subset diffs [ Slug_f; Rank ] + && (not (List.mem Slug_f diffs) || List.mem c.slug nativity_octave_day_slugs) + then Some "C6" else if (String.equal c.slug "matthew" || String.equal c.slug "thomas") && subset diffs [ Slug_f; Colour_f ] then Some "C7" else if @@ -369,7 +444,10 @@ let layer_c_reason (l : row) (c : row) diffs = || String.equal c.slug "joseph-spouse-of-the-bl-virgin-mary") && subset diffs [ Season; Slug_f; Rank; Colour_f ] then Some "C9" - else if (String.equal l.date "2011-07-02" || String.equal l.date "2011-07-04") then Some "C10" + else if + (String.equal l.date "2011-07-02" || String.equal l.date "2011-07-04") + && subset diffs [ Slug_f; Rank; Colour_f ] + then Some "C10" else None (* ---------------------------------------------------------------------- *) @@ -423,6 +501,16 @@ let describe_unexplained (l : row) (c : row) diffs = (String.concat "," (List.map field_name diffs)) l.weekday l.season l.slug l.rank l.colour c.weekday c.season c.slug c.rank c.colour +(* Fix round 1, finding 4: the fixture's own byte content must still match + the SHA-256 its provenance note claims (lectio commit 2386a45, recorded + both here and in test/fixtures/lectio-ef-2005-2050.provenance). Checked + before anything else reads the fixture -- a drifted fixture makes every + other assertion in this suite a statement about an unlabelled snapshot, + not about the pinned lectio commit it claims to be. *) +let test_fixture_checksum () = + Alcotest.(check string) "fixture SHA-256 matches its provenance note" fixture_sha256 + (sha256_of_file fixture_path) + (* Dates align 1:1 in the same order on both streams (both are one line per civil day, 2005-01-01..2050-12-31 -- see the fixture's own provenance note and [colitur_rows_2005_2050]'s construction). A silent misalignment @@ -486,7 +574,8 @@ let test_layer_c_counts_match_citations () = let suite = ( "differential (lectio, EF, 2005-2050)", - [ Alcotest.test_case "streams are 16801 rows each, dates aligned 1:1" `Quick test_dates_align; + [ Alcotest.test_case "fixture SHA-256 matches its provenance note" `Quick test_fixture_checksum; + Alcotest.test_case "streams are 16801 rows each, dates aligned 1:1" `Quick test_dates_align; Alcotest.test_case "every difference is normalised (A/B) or cited (C) -- none unexplained" `Quick test_no_unexplained_differences; Alcotest.test_case "Layer C counts match data/ef/expected-divergences.sexp exactly" `Quick -- cgit v1.3 From c464c0435c650cf019eb6c6ae1906217d557a15f Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 12 Aug 2026 04:12:53 +0200 Subject: kernel(temporal): Holy Thursday is white, per RG128(b)'s named exception MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit RG 128(b) excepts "Missa sive Chrismatis sive in Cena Domini feria V Hebdomadae sanctae" from the Septuagesima-to-Easter-Vigil violet span as a whole-Mass exception (unlike Palm Sunday's blessing/procession, which the same sentence carves out as only part of that day) -- so Holy Thursday's Mass is white, not violet. temporal_ef.ml fell through to the generic Passiontide ferial path (season_colour Passiontide = violet) for this date, since [named] has no entry for the Triduum at all (rank was already correct via [privileged_feria]; only colour was wrong). Found via Task 16's missalemeum oracle comparison: every other Triduum day's oracle colour set includes violet as an option (Good Friday "bv", Holy Saturday "vw" -- already-flagged, deliberately deferred per-action gaps, register §3b), but Holy Thursday's is white alone. This is a genuine NEW divergence from lectio too (lectio has no such exception and still prints violet for Holy Thursday) -- added as Layer C11 to the differential harness and data/ef/expected-divergences .sexp, 46 rows (2005-2050), rather than silently changing what that harness accepts. Also corrected a stale comment on season_colour claiming white's RG paragraph was never pinned -- RG 119 was found and cited in the register on 2026-08-11 but the correction was never copied into this file's own comment. --- data/ef/expected-divergences.sexp | 5 +++++ lib/rites/rite_ef/temporal_ef.ml | 33 ++++++++++++++++++++++++++++----- test/test_differential.ml | 16 +++++++++------- 3 files changed, 42 insertions(+), 12 deletions(-) (limited to 'test/test_differential.ml') diff --git a/data/ef/expected-divergences.sexp b/data/ef/expected-divergences.sexp index 17bafc1..6b7bbdd 100644 --- a/data/ef/expected-divergences.sexp +++ b/data/ef/expected-divergences.sexp @@ -67,3 +67,8 @@ (verdict colitur) (note "2011: Sacred Heart (Friday after the Corpus Christi octave) falls on 1 July and impedes the Precious Blood (also 1 July, I class but a lower RG 91 entry). RG 96's walk must skip 2 July (Visitation, II class) and 3 July (Sunday), landing on 4 July (Monday, a free class-4 feria). lectio instead lands Precious Blood on 2 July, illegitimately displacing the Visitation -- exactly what RG 96's \"not I or II class\" clause forbids.") (expected_rows 2)) + ((id C11) + (citation "RG 128(b), primary-source-verified 2026-08-12 (Task 16): \"...a dominica in Septuagesima usque ad Vigiliam paschalem, exceptis: ... Missa sive Chrismatis sive in Cena Domini feria V Hebdomadae sanctae; ...\" -- violet runs Septuagesima to the Easter Vigil EXCEPT (among other things) the Mass of Holy Thursday, named as a whole-Mass exception") + (verdict colitur) + (note "Holy Thursday (\"ef-passiontide-2-thursday\") is white in colitur, per RG 128(b)'s explicit exception -- independently confirmed by the missalemeum oracle (Task 16 report), whose colour set for that day is white alone, unlike the surrounding Triduum days which include violet as an option. lectio has no such exception and prints Passiontide's base violet straight through Holy Thursday. Found via Task 16's oracle comparison, not the original lectio differential (lectio's own colour did not previously disagree, because colitur's own bug matched it) -- fixed in colitur, so this row records a NEW divergence from lectio's still-uncorrected violet, not a pre-existing one.") + (expected_rows 46)) diff --git a/lib/rites/rite_ef/temporal_ef.ml b/lib/rites/rite_ef/temporal_ef.ml index 52a9adb..b39da79 100644 --- a/lib/rites/rite_ef/temporal_ef.ml +++ b/lib/rites/rite_ef/temporal_ef.ml @@ -314,13 +314,15 @@ let privileged_feria d = (n >= -6 && n <= -1) || (n >= 1 && n <= 6) || (n >= 50 && n <= 55) (* RG 117 enumerates the five colours (white, red, green, violet, black); - RG 127 assigns green and RG 128 violet to the seasons de Tempore below. - White's own specific paragraph (the "B) De colore albo" section, between - 117 and 123) was not pinned by the primary-source search available here -- - left uncited rather than guessed; see register §3 "Colours". *) + RG 127 assigns green and RG 128 violet to the seasons de Tempore below. RG + 119 (register §3b, primary-source-verified 2026-08-11 -- this comment was + stale until Task 16 noticed the correction had not been copied down here): + white "a festo Nativitatis Domini usque ad expletum tempus Epiphaniae" + and "a Missa Vigiliae paschalis usque ad Missam vigiliae Pentecostis + exclusive" -- exactly Christmastide and Paschaltide below. *) let season_colour = function | Advent | Septuagesima | Lent | Passiontide -> Colour.Violet (* RG 128 *) - | Christmastide | Paschaltide -> Colour.White + | Christmastide | Paschaltide -> Colour.White (* RG 119 *) | Time_after_epiphany | Time_after_pentecost -> Colour.Green (* RG 127 *) (* Gaudete (Advent III) and Laetare (Lent IV) are rose: RG 131, "may be used... @@ -411,6 +413,27 @@ let temporal d = let colour = (* The Pentecost octave weekdays are red, not Paschaltide's white. *) if days_between easter d >= 50 && days_between easter d <= 55 then Colour.Red + (* RG 128(b) (docs/research/rules-register.md §3b), primary + text: "...a dominica in Septuagesima usque ad Vigiliam + paschalem, EXCEPTIS: ... MISSA SIVE CHRISMATIS SIVE IN + CENA DOMINI FERIA V HEBDOMADAE SANCTAE; ..." -- violet + runs Septuagesima to the Easter Vigil EXCEPT (among + others) "the Mass, whether of the Chrism or in Cena + Domini [Holy Thursday], on Thursday of Holy Week" -- + named as a WHOLE-MASS exception (unlike Palm Sunday's + "blessing and procession of palms", which the SAME + sentence carves out as only PART of that day, register + §3b's own RG126 note on the not-yet-modelled per-action + nuance), so this is a clean whole-day colour fact, not + a per-action one the day/colour model cannot express. + Task 16, found via the missalemeum oracle comparison: + every other Triduum day's oracle colour SET includes + violet as one option (Good Friday "bv", Holy Saturday + "vw" -- RG 132's black is a separate, ALREADY-flagged + gap, register §3b, not touched here), but Holy + Thursday's is white ALONE -- confirming this specific + day, and only this one, needs the exception coded. *) + else if days_between easter d = -3 then Colour.White else season_colour s in let week_n = week d in diff --git a/test/test_differential.ml b/test/test_differential.ml index dd601a1..a9bdb91 100644 --- a/test/test_differential.ml +++ b/test/test_differential.ml @@ -76,13 +76,14 @@ family would still be caught by everything except the digit itself. - Layer C (data/ef/expected-divergences.sexp, matched by - [layer_c_reason] below): the CITED allow-list. Ten genuine liturgical - disagreements, each citing its RG paragraph and stating which engine is - right (always colitur, verified against the Missal/register, never - against lectio's own behaviour -- "lectio does it differently" is not - itself a justification anywhere in this file). This is the ONLY layer - that may cover a difference in rank, colour, or which celebration is - observed; A and B never do (enforced structurally below: A/B only ever + [layer_c_reason] below): the CITED allow-list. Eleven genuine liturgical + disagreements (C11 added by Task 16's oracle work, below), each citing + its RG paragraph and stating which engine is right (always colitur, + verified against the Missal/register, never against lectio's own + behaviour -- "lectio does it differently" is not itself a justification + anywhere in this file). This is the ONLY layer that may cover a + difference in rank, colour, or which celebration is observed; A and B + never do (enforced structurally below: A/B only ever touch the season/slug fields, and Layer C's predicates each require an exact, narrow field-diff SET, not "anything goes"). @@ -448,6 +449,7 @@ let layer_c_reason (l : row) (c : row) diffs = (String.equal l.date "2011-07-02" || String.equal l.date "2011-07-04") && subset diffs [ Slug_f; Rank; Colour_f ] then Some "C10" + else if String.equal c.slug "ef-passiontide-2-thursday" && diffs = [ Colour_f ] then Some "C11" else None (* ---------------------------------------------------------------------- *) -- cgit v1.3