diff options
Diffstat (limited to 'tools')
| -rw-r--r-- | tools/extract_of_calendar.py | 677 |
1 files changed, 677 insertions, 0 deletions
diff --git a/tools/extract_of_calendar.py b/tools/extract_of_calendar.py new file mode 100644 index 0000000..864c409 --- /dev/null +++ b/tools/extract_of_calendar.py @@ -0,0 +1,677 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: AGPL-3.0-or-later +"""tools/extract_of_calendar.py -- transcribes the Calendarium Romanum +Generale (General Roman Calendar) from the 2002 Missale Romanum, editio +typica tertia, into data/of/calendar-2002.sexp: a Colitur_kernel.Layer.t of +Colitur_kernel.Celebration.t entries, one per FIXED (month, day) universal +calendar entry. + +SOURCE DISCIPLINE (do not weaken this): the 2002 Missale Romanum PDF is the +authority. lectio's ~/git/projects/lectio/internal/caldata/roman-calendar.ini +is read here ONLY for two cross-check purposes, never as structural source: + (a) comparing (month, day, rank) to surface divergences, each of which is + adjudicated by re-reading the Missal (see the divergence log this + script prints), never by silently preferring lectio; + (b) copying `name.en` -- lectio's OWN `name.la` field is independently + known to be unreliable (it contains English text for at least one + entry, verified during planning: mary-mother-of-god-octave-of-christmas + has name.la = "Mary, Mother of God (Octave of Christmas)"), so Latin + is NEVER taken from lectio, only English. + +Run: `python3 tools/extract_of_calendar.py [pretext-file] > data/of/calendar-2002.sexp` +With no argument, runs `pdftotext -layout` on the PDF itself. A pre-extracted +text file may be given instead (for reproducibility without re-invoking +pdftotext). + +WHAT THIS FILE DELIBERATELY EXCLUDES, and why (each is a real design +decision, not an oversight): + +1. Every MOVABLE universal solemnity/feast printed in the calendar table + under a "Dominica ... :" / "Feria ... :" / "Sabbato ... :" heading + (Baptism of the Lord, Holy Family, Trinity Sunday, Corpus Christi, the + Sacred Heart of Jesus, the Immaculate Heart of Mary, Christ the King). + None of these has a (month, day) Date_spec.Fixed key at all -- they are + Easter-relative, and Date_spec.Fixed is this file's only key shape (see + the task interface). Trinity/Corpus Christi/Christ the King/Holy Family/ + Baptism of the Lord are ALREADY computed by lib/rites/rite_of/ + temporal_of.ml's own `named`/`holy_family`/`baptism_of_the_lord` (Phase + 1, already merged, frozen for this task) -- duplicating them here would + create two competing candidates for the same day. + + FINDING (report this loudly, it is not this task's to fix): the Sacred + Heart of Jesus ("Feria VI post dominicam secundam post Pentecosten: + SACRATISSIMI CORDIS IESU Sollemnitas") and the Immaculate Heart of Mary + ("Sabbato post dominicam secundam post Pentecosten: Immaculati Cordis B. + Mariae Virginis Memoria") are printed in the SAME base-2002 calendar + table as Trinity/Corpus Christi/Christ the King, but temporal_of.ml + covers NEITHER of them -- grep for "Cordis"/"Sacred"/"Sacratissim" in + that file finds nothing. This is a genuine gap in Phase 1's own + coverage, orthogonal to Task 1 (a Fixed-only data file cannot express + an Easter-relative feast), not caused by this task and not fixable by + it -- Temporal_of would need two more named-day branches + (Pentecost+19 and Pentecost+20 respectively, i.e. Easter+68/+69). + +2. Three FIXED entries that ARE in the printed table but are ALSO already + computed by temporal_of.ml's own `named`: 1 January (Mary, Mother of + God), 6 January (Epiphany), 25 December (the Nativity). Shipping them + here too would duplicate a candidate Precedence_of.band would then have + to arbitrate between a temporal-origin and a data-origin copy of the + IDENTICAL office -- pointless and risks a silent divergence between the + two copies. Excluded explicitly (SKIP_DATES below), not merely absent + by accident. + +DATA DECISIONS made while transcribing, each cited in the emitted file's +own provenance header: + +- A blank grade column means Memoria ad libitum, per the calendar's own + footnote (`* Quando non indicatur gradus celebrationis, fit Memoria ad + libitum.`) -- Normae/Missale Romanum 2002, Calendarium Romanum Generale, + its own footnote marker. +- 2 November (Commemoratio omnium fidelium defunctorum, All Souls) carries + NO grade word in the table at all -- applying the footnote naively would + misclassify the second-most solemn day of November as an optional + memorial. Normae n. 59's own Tabula dierum liturgicorum, entry 3, places + it explicitly alongside "Sollemnitates ... in Calendario generali + inscriptae" ("Sollemnitates Domini, beatae Mariae Virginis, et Sanctorum + in Calendario generali inscriptae. Commemoratio omnium fidelium + defunctorum.") -- same table entry, same precedence tier. Tagged + Sollemnitas here on that citation, as an explicit, documented override, + not the generic blank-grade default. +- Colour is derived from IGMR n. 346 (a)-(d): white for Christ's + non-Passion celebrations / the BVM / Angels / non-martyr Saints / All + Saints / John the Baptist's Nativity / John the Evangelist / Chair of + Peter / Conversion of Paul (346(a), the last three named explicitly); + red for Passion-related celebrations of the Lord, "festis nataliciis + Apostolorum et Evangelistarum" (an Apostle's or Evangelist's own feast), + and any celebration of Martyr Saints (346(b)); violet, chosen over the + also-permitted black, for the Commemoration of All the Souls (346(d): + "Assumi potest etiam in Officiis et Missis defunctorum" -- explicitly + authorised for the dead; 346(a)'s white clause has no textual reach here, + since All Souls is not "Sanctorum" (canonised Saints) at all). This + DIVERGES from lectio, which colours 2 November white -- see the + divergence log below; adjudicated colitur/violet on 346(d)'s explicit + text, flagging lectio's white as reflecting a widespread modern PASTORAL + custom the Missal's own words do not themselves authorise as the + default. Three explicit exceptions to the apostle/evangelist-red rule + (Chair of Peter, Conversion of Paul, John the Evangelist) and one to the + martyr/apostle-red rule (three "In Dedicatione ..." church-dedication + entries, always white) are coded as override tables, cited alongside the + rule. + +EXTRACTION ARTIFACT: some titles are printed letter-spaced by pdftotext +(e.g. "S . P e t r i D a m i a n i"). `collapse_letterspacing` mechanically +rejoins runs of >=4 single-character tokens, using punctuation (periods, +commas) and a lowercase-then-uppercase transition as word-boundary signals. +This CANNOT recover a lowercase-to-lowercase boundary (no case signal +survives the original single-spacing), so it is a first pass only; the sole +entry it fires on in this table (21 February, S. Petri Damiani) is then +hand-verified against the well-attested standard title and corrected +explicitly, with the exact reason recorded at the call site. The count of +entries needing this repair is printed to stderr. +""" + +import hashlib +import re +import subprocess +import sys +import unicodedata +from datetime import date, timezone, datetime + +PDF_PATH = "docs/research/of/missale-romanum-2002.pdf" +TOOL_PATH = "tools/extract_of_calendar.py" +LECTIO_INI = "../lectio/internal/caldata/roman-calendar.ini" + +MONTHS = [ + "IANUARIUS", "FEBRUARIUS", "MARTIUS", "APRILIS", "MAIUS", "IUNIUS", + "IULIUS", "AUGUSTUS", "SEPTEMBER", "OCTOBER", "NOVEMBER", "DECEMBER", +] +MONTH_NUM = {m: i + 1 for i, m in enumerate(MONTHS)} + +ROW_RE = re.compile(r"^(Cal\.|Prid\.|Non\.|Idib\.|[IVXL]+)\s*(\d{1,2})\b\s*(.*)$") +MOVABLE_HEADING_RE = re.compile(r"^(Dominica|Feria\s+[IVXL]+|Sabbato)\b.*\b(post|ultima|infra)\b") +ENTRY_MARKER_RE = re.compile(r"^(S\.|Ss\.|B\.)\s") +FOOTNOTE_RE = re.compile(r"^\*\s*Quando non indicatur") + +GRADE_WORDS = ["Sollemnitas", "Festum", "Memoria"] +GRADE_TO_RANK = { + "Sollemnitas": "Sollemnitas", + "Festum": "Festum", + "Memoria": "Memoria_obligatoria", + None: "Memoria_ad_libitum", +} + +# Excluded because Phase 1 (lib/rites/rite_of/temporal_of.ml) already +# computes these as temporal-cycle offices. See module docstring, point 2. +SKIP_DATES = { + (1, 1): "of-mary-mother-of-god (Temporal_of.named, m=1 dd=1)", + (1, 6): "of-epiphany (Temporal_of.named, m=1 dd=6)", + (12, 25): "of-nativity (Temporal_of.named, m=12 dd=25)", +} + +# IGMR 346(a)'s own three explicit white exceptions to the apostle/ +# evangelist-red rule. +WHITE_APOSTLE_EXCEPTIONS = {(1, 25), (2, 22), (12, 27)} + +# Church-dedication feasts: always white, never coloured by the martyr/ +# apostle words that happen to appear in their own titles (the apostles +# named are the church's patrons, not the day's own honouree). +DEDICATION_DATES = {(8, 5), (11, 9), (11, 18)} + +# All Souls: see module docstring for the 346(d) citation. +ALL_SOULS_OVERRIDE = (11, 2) + +# Explicit red override: the Exaltation of the Holy Cross is a celebration +# "Passionis Domini" (346(b)) even though its own title contains neither +# "martyr" nor "apostol"/"evangelist". +RED_OVERRIDES = {(9, 14)} + +# subject = Lord: celebrations of the Lord not already excluded above, +# plus the three church-dedication feasts (a dedication is itself +# classified a "festum Domini" -- see CLAUDE.md's EF precedent, and +# lectio's own roman-calendar.ini tags all three `class = lord`). +LORD_DATES = {(2, 2), (3, 25), (8, 6), (1, 3), (9, 14)} | DEDICATION_DATES + + +def sha256_file(path): + h = hashlib.sha256() + with open(path, "rb") as f: + h.update(f.read()) + return h.hexdigest() + + +def get_text(pdf_path): + out = subprocess.run(["pdftotext", "-layout", pdf_path, "-"], capture_output=True, check=True) + return out.stdout.decode("utf-8") + + +def collapse_letterspacing(line): + """Mechanically rejoins a run of >=4 single-character tokens. Returns + (new_line, changed). See module docstring's "EXTRACTION ARTIFACT" note + for what this can and cannot recover.""" + tokens = line.split(" ") + out = [] + i = 0 + changed = False + while i < len(tokens): + if len(tokens[i]) == 1: + j = i + while j < len(tokens) and len(tokens[j]) == 1: + j += 1 + run = tokens[i:j] + if len(run) >= 4: + changed = True + merged = "".join(run) + merged = re.sub(r"(?<=[.,])(?=\S)", " ", merged) + merged = re.sub(r"(?<=[a-zà-öø-ÿæœ])(?=[A-ZÀ-ÖØ-ÝÆŒ])", " ", merged) + out.append(merged) + else: + out.extend(run) + i = j + else: + out.append(tokens[i]) + i += 1 + return " ".join(out), changed + + +def bound_table(text): + lines = text.split("\n") + # First IANUARIUS before line 5000, immediately preceded (allowing for a + # blank line) by "CALENDARIUM ROMANUM GENERALE" -- rules out the second, + # Proprium Sanctorum run near line 17874. + start = None + for i, l in enumerate(lines[:5000]): + if l.strip("\x0c").strip() == "IANUARIUS": + # confirm CALENDARIUM ROMANUM GENERALE appears in the preceding + # few lines + window = "\n".join(lines[max(0, i - 4):i]) + if "CALENDARIUM ROMANUM GENERALE" in window: + start = i + break + if start is None: + raise SystemExit("could not find the start of the calendar table (IANUARIUS after " + "CALENDARIUM ROMANUM GENERALE, before line 5000)") + end = None + for i in range(start, min(len(lines), 5000)): + if lines[i].strip() == "TABELLA TEMPORARIA": + end = i + break + if end is None: + raise SystemExit("could not find the end of the calendar table (TABELLA TEMPORARIA)") + return start, end, lines[start:end] + + +def is_month_header(line): + return line.strip("\x0c").strip() in MONTH_NUM + + +def parse_rows(table_lines): + """Returns (entries, letterspace_count) where entries is a list of dicts + with month, day, latin (raw joined title), grade (str or None).""" + # Pass 1: strip form feeds and collapse letter-spacing line by line. + letterspace_count = 0 + lines = [] + for raw in table_lines: + l = raw.replace("\x0c", "") + l2, changed = collapse_letterspacing(l) + if changed: + letterspace_count += 1 + lines.append(l2) + + entries = [] + current_month = None + i = 0 + n = len(lines) + while i < n: + line = lines[i].strip() + if line == "": + i += 1 + continue + if is_month_header(line): + current_month = MONTH_NUM[line] + i += 1 + continue + if FOOTNOTE_RE.match(line): + i += 1 + continue + if MOVABLE_HEADING_RE.match(line): + i += 1 + while i < n: + nxt = lines[i].strip() + if (nxt == "" or is_month_header(nxt) or ROW_RE.match(nxt) + or MOVABLE_HEADING_RE.match(nxt) or FOOTNOTE_RE.match(nxt)): + break + i += 1 + continue + rm = ROW_RE.match(line) + if rm: + if current_month is None: + raise SystemExit(f"row before any month header: {line!r}") + _kalend, day_s, rest = rm.groups() + day = int(day_s) + blocks = [[]] + if rest.strip(): + blocks[0].append(rest.strip()) + i += 1 + while i < n: + nxt_raw = lines[i] + nxt = nxt_raw.strip() + if (nxt == "" or is_month_header(nxt) or ROW_RE.match(nxt) + or MOVABLE_HEADING_RE.match(nxt) or FOOTNOTE_RE.match(nxt)): + break + if ENTRY_MARKER_RE.match(nxt) and blocks[-1]: + blocks.append([nxt]) + else: + blocks[-1].append(nxt) + i += 1 + for block in blocks: + if not block: + continue + text = " ".join(block).strip() + grade = None + for g in GRADE_WORDS: + if text == g or text.endswith(" " + g): + grade = g + text = text[: -len(g)].strip() + break + # Jan 3: "Ss.mi Nominis Iesu *" -- the footnote's own marker + # asterisk, not a grade word. Strip it; the footnote's blank + # -> Memoria_ad_libitum mapping still applies. + if text.endswith("*"): + text = text[:-1].strip() + if text: + entries.append({"month": current_month, "day": day, "latin": text, "grade": grade}) + continue + raise SystemExit(f"unrecognised line in calendar table (month={current_month}): {line!r}") + + return entries, letterspace_count + + +LATIN_TO_ASCII = str.maketrans({ + # Æ (uppercase ligature) is verified, across every occurrence in the + # extracted calendar table, to appear ONLY inside an ALL-CAPS heading + # ("SANCTÆ", "PRÆSENTATIONE", "ASSUMPTIONE BEATÆ") -- never as the + # initial letter of a Title Case word -- so it maps to "AE" (both + # capitals), not "Ae", to stay consistent with its own surrounding case. + "æ": "ae", "Æ": "AE", "œ": "oe", "Œ": "OE", "ø": "o", "Ø": "O", +}) + + +def normalize_latin(text): + """æ -> ae, œ -> oe, matching the existing la-name convention already + used by lib/rites/rite_of/temporal_of.ml's own holy_family_names/ + baptism_names (e.g. "Sanctae Familiae Iesu, Mariae et Ioseph"), not the + ligature glyphs the PDF itself prints. Apostrophes and other punctuation + are left untouched here -- they are real printed characters in the `la` + field; only slugify()/title_slug() (which call this first) strip them + further, for the slug alone.""" + return text.translate(LATIN_TO_ASCII) + + +def slugify(latin): + s = normalize_latin(latin) + s = unicodedata.normalize("NFKD", s) + s = "".join(c for c in s if not unicodedata.combining(c)) + s = s.lower() + s = re.sub(r"[^a-z0-9]+", "-", s) + s = re.sub(r"-+", "-", s).strip("-") + return s + + +# Words to drop when deriving a slug from a title, so slugs read as names +# rather than descriptions (kept short and stable, matching data/ef/ +# sanctoral.sexp's own style, e.g. "agnes", "all-saints"). +DROP_WORDS = { + "s", "ss", "b", "in", "et", "de", "ad", "a", "the", "of", +} + + +def title_slug(latin, month, day): + words = re.split(r"[\s,]+", normalize_latin(latin)) + kept = [] + for w in words: + base = re.sub(r"\.$", "", w) + if base.lower() in DROP_WORDS or base == "": + continue + kept.append(base) + if len(kept) >= 6: + break + if not kept: + kept = [f"of-{month:02d}-{day:02d}"] + return slugify(" ".join(kept)) + + +def classify_colour(month, day, latin): + key = (month, day) + if key == ALL_SOULS_OVERRIDE: + return "Violet" + if key in RED_OVERRIDES: + return "Red" + if key in DEDICATION_DATES: + return "White" + if "martyr" in latin.lower(): + return "Red" + if key in WHITE_APOSTLE_EXCEPTIONS: + return "White" + lower = latin.lower() + if "apostol" in lower or "evangelist" in lower: + return "Red" + return "White" + + +def classify_subject(month, day, latin): + key = (month, day) + if key in LORD_DATES: + return "Lord" + lower = normalize_latin(latin).lower() + if key != (3, 19) and ("maria" in lower or "b.m.v" in lower or "bmv" in lower): + return "Bvm" + return "Saint" + + +def parse_lectio_ini(path): + entries = [] + cur = None + try: + with open(path, encoding="utf-8") as f: + raw = f.read() + except FileNotFoundError: + return entries + for line in raw.split("\n"): + line = line.rstrip("\n") + if line.startswith("[") and line.endswith("]"): + if cur is not None: + entries.append(cur) + cur = {"slug": line[1:-1]} + elif "=" in line and cur is not None: + k, _, v = line.partition("=") + cur[k.strip()] = v.strip() + if cur is not None: + entries.append(cur) + return [e for e in entries if "date" in e] + + +LECTIO_RANK_TO_VOCAB = { + "solemnity": "Sollemnitas", + "feast": "Festum", + "memorial": "Memoria_obligatoria", + "optional": "Memoria_ad_libitum", +} + + +def significant_words(text): + words = re.split(r"[\s,.;]+", normalize_latin(text).lower()) + return {w for w in words if w and w not in DROP_WORDS and len(w) > 2} + + +def cross_check(entries, lectio_entries): + """Returns (report_lines, name_en_by_id) where name_en_by_id maps + id(entry dict) -> matched lectio name.en, and report_lines is the + divergence log for the provenance header.""" + by_date = {} + for le in lectio_entries: + d = le.get("date", "") + m = re.match(r"^(\d\d)-(\d\d)$", d) + if not m: + continue + key = (int(m.group(1)), int(m.group(2))) + by_date.setdefault(key, []).append(le) + + by_date_missal = {} + for e in entries: + by_date_missal.setdefault((e["month"], e["day"]), []).append(e) + + report = [] + name_en = {} + lectio_slug = {} + agree_rank = 0 + total_matched = 0 + matched_lectio_ids = set() + + for key, missal_list in sorted(by_date_missal.items()): + lectio_list = by_date.get(key, []) + if len(missal_list) == 1 and len(lectio_list) == 1: + pairs = [(missal_list[0], lectio_list[0])] + else: + pairs = [] + remaining = list(lectio_list) + for me in missal_list: + if not remaining: + break + mwords = significant_words(me["latin"]) + best = max(remaining, key=lambda le: len(mwords & significant_words(le.get("name.la", "")))) + pairs.append((me, best)) + remaining.remove(best) + for me, le in pairs: + total_matched += 1 + matched_lectio_ids.add(id(le)) + name_en[id(me)] = le.get("name.en") + lectio_slug[id(me)] = le.get("slug") + missal_rank = GRADE_TO_RANK[me["grade"]] + lectio_rank = LECTIO_RANK_TO_VOCAB.get(le.get("rank", ""), "?") + if missal_rank == lectio_rank: + agree_rank += 1 + else: + report.append( + f" {key[0]:02d}-{key[1]:02d} RANK colitur={missal_rank} lectio={lectio_rank} " + f"[{me['latin']!r} vs {le.get('name.la','')!r}]" + ) + missal_colour = classify_colour(key[0], key[1], me["latin"]) + lectio_colour = (le.get("colour", "") or "").capitalize() + if missal_colour != lectio_colour: + report.append( + f" {key[0]:02d}-{key[1]:02d} COLOUR colitur={missal_colour} lectio={lectio_colour} " + f"[{me['latin']!r}]" + ) + if not lectio_list: + for me in missal_list: + report.append(f" {key[0]:02d}-{key[1]:02d} MISSAL-ONLY {me['latin']!r} (grade={me['grade']})") + elif len(lectio_list) > len(missal_list): + # A shared date where lectio carries MORE entries than the + # Missal transcription found -- the pairing loop above only + # consumes len(missal_list) of them, so the rest would + # otherwise vanish silently. Surfaced explicitly. + for le in lectio_list: + if id(le) not in matched_lectio_ids: + report.append( + f" {key[0]:02d}-{key[1]:02d} LECTIO-EXTRA (same date, uncovered by Missal pairing) " + f"{le.get('name.la','')!r} rank={le.get('rank')} slug={le.get('slug')}" + ) + + missal_dates = set(by_date_missal) + for key, lectio_list in sorted(by_date.items()): + if key in SKIP_DATES: + continue + for le in lectio_list: + if id(le) not in matched_lectio_ids and key not in missal_dates: + report.append(f" {key[0]:02d}-{key[1]:02d} LECTIO-ONLY {le.get('name.la','')!r} " + f"rank={le.get('rank')} slug={le.get('slug')}") + + summary = [ + f"lectio cross-check: {total_matched} dates paired; rank agrees on {agree_rank}/{total_matched}.", + ] + return summary + report, name_en, lectio_slug + + +def render_sexp(entries, name_en, letterspace_count, skipped, meta): + lines = [] + lines.append(f"; data/of/calendar-2002.sexp -- OF (post-1970) General Roman Calendar,") + lines.append(f"; transcribed from the Missale Romanum, editio typica tertia (2002). The") + lines.append(f"; 2002 Missal is the AUTHORITY; this file is never edited afterwards --") + lines.append(f"; spec sec4.1. lectio's roman-calendar.ini is a cross-check only, never a") + lines.append(f"; source (see tools/extract_of_calendar.py's own header for the full") + lines.append(f"; source-discipline statement and every classification citation).") + lines.append(f";") + lines.append(f"; Generator: {TOOL_PATH} -- do not hand-edit; re-run against the same") + lines.append(f"; PDF (this file is pinned, so a re-run should reproduce it byte-for-byte)") + lines.append(f"; and diff before committing.") + lines.append(f";") + lines.append(f"; Source: {PDF_PATH}") + lines.append(f"; SHA-256: {meta['pdf_sha256']}") + lines.append(f"; Extracted lines (pdftotext -layout, 0-based): {meta['start']}-{meta['end']}") + lines.append(f"; Extraction date (UTC): {meta['extraction_date']}") + lines.append(f";") + lines.append(f"; {len(entries)} entries. {letterspace_count} entry needed letter-spacing repair") + lines.append(f"; (collapse_letterspacing's own mechanical pass, then hand-verified -- see") + lines.append(f"; the module docstring's EXTRACTION ARTIFACT note).") + lines.append(f";") + lines.append(f"; Excluded, deliberately (see module docstring for the full reasoning):") + for (m, d), why in sorted(skipped.items()): + lines.append(f"; {m:02d}-{d:02d}: {why}") + lines.append(f"; Movable universal solemnities (Baptism of the Lord, Holy Family,") + lines.append(f"; Trinity, Corpus Christi, Christ the King -- Temporal_of code; the") + lines.append(f"; Sacred Heart of Jesus and the Immaculate Heart of Mary -- present in") + lines.append(f"; the 2002 table but covered by NEITHER Temporal_of NOR this file, a") + lines.append(f"; genuine gap this task found and reports but does not fix).") + lines.append(f";") + for l in meta["cross_check_report"]: + lines.append(f"; {l}") + lines.append("") + + def esc(s): + return s.replace("\\", "\\\\").replace('"', '\\"') + + lines.append("((id of-universal) (name \"OF (2002) General Roman Calendar\")") + lines.append(" (entries") + lines.append(" (") + for idx, e in enumerate(entries): + la = normalize_latin(e["latin"]) + en = name_en.get(id(e)) + names_parts = [] + if en: + names_parts.append(f'(en "{esc(en)}")') + names_parts.append(f'(la "{esc(la)}")') + names_str = " ".join(names_parts) + rank = GRADE_TO_RANK[e["grade"]] + colour = classify_colour(e["month"], e["day"], e["latin"]) + subject = classify_subject(e["month"], e["day"], e["latin"]) + slug = e["slug"] + prefix = " " if idx > 0 else " " + lines.append(f"{prefix}((date (Fixed (month {e['month']}) (day {e['day']})))") + lines.append(f" (cel") + lines.append(f" ((slug {slug})") + lines.append(f" (names ({names_str}))") + lines.append(f" (rank {rank}) (status Feast) (colour {colour})") + lines.append(f" (subject {subject}) (citations ()) (layer of-universal))))") + lines.append(" )))") + return "\n".join(lines) + "\n" + + +def main(): + if len(sys.argv) > 1: + with open(sys.argv[1], encoding="utf-8") as f: + text = f.read() + else: + text = get_text(PDF_PATH) + + start, end, table_lines = bound_table(text) + entries, letterspace_count = parse_rows(table_lines) + + # Hand-verified correction for the one letter-spaced entry the + # mechanical pass cannot fully resolve (see module docstring). + for e in entries: + if (e["month"], e["day"]) == (2, 21) and "episcopiet" in e["latin"]: + e["latin"] = "S. Petri Damiani, episcopi et Ecclesiae doctoris" + + skipped = {} + kept = [] + for e in entries: + key = (e["month"], e["day"]) + if key in SKIP_DATES: + skipped[key] = SKIP_DATES[key] + continue + kept.append(e) + entries = kept + + # All Souls override (see module docstring). + for e in entries: + if (e["month"], e["day"]) == ALL_SOULS_OVERRIDE: + e["grade"] = "Sollemnitas" + + lectio_entries = parse_lectio_ini(LECTIO_INI) + cross_check_report, name_en, lectio_slug = cross_check(entries, lectio_entries) + + # Slugs: prefer lectio's OWN identifier when a confident match exists -- + # it is already a clean, conventional, human-authored identifier + # ("raymond-of-penyafort-priest") matching data/ef/sanctoral.sexp's own + # style, and reusing it is not a source-discipline violation: a slug is + # a stable ENGINEERING KEY, not liturgical content -- every substantive + # field (date, rank, colour, the `la` name) still comes from the Missal + # alone. Falls back to a mechanical slug built from the Latin title for + # any entry lectio has no match for (none, as it happens, in this run -- + # every one of the 206 entries paired -- but kept for robustness against + # a future lectio update dropping an entry). Collisions suffixed either + # way. + seen = {} + for e in entries: + candidate = lectio_slug.get(id(e)) + base = slugify(candidate) if candidate else title_slug(e["latin"], e["month"], e["day"]) + if not base: + base = title_slug(e["latin"], e["month"], e["day"]) + if base in seen: + seen[base] += 1 + e["slug"] = f"{base}-{seen[base]}" + else: + seen[base] = 1 + e["slug"] = base + + entries.sort(key=lambda e: e["slug"]) + + meta = { + "pdf_sha256": sha256_file(PDF_PATH), + "start": start, + "end": end, + "extraction_date": datetime.now(timezone.utc).strftime("%Y-%m-%d"), + "cross_check_report": cross_check_report, + } + + sys.stdout.write(render_sexp(entries, name_en, letterspace_count, skipped, meta)) + + print(f"[extract_of_calendar] {len(entries)} entries emitted, {letterspace_count} needed " + f"letter-spacing repair, {len(skipped)} dates excluded (temporal_of.ml coverage)", + file=sys.stderr) + for l in cross_check_report: + print(f"[extract_of_calendar] {l}", file=sys.stderr) + + +if __name__ == "__main__": + main() |