diff options
| author | Lukasz Kasprzak <lukas@labunix.xyz> | 2026-08-19 15:01:18 +0200 |
|---|---|---|
| committer | Lukasz Kasprzak <lukas@labunix.xyz> | 2026-08-19 15:01:18 +0200 |
| commit | 7f263a0ec9a91d1a036cfd22ed38354d06500b1d (patch) | |
| tree | 6772dea9f35c269ed6a5fda8e2dd30f306b2325b /Makefile | |
| parent | 22824ef50abfeb497f04e73f88d0f7ab2e164eaf (diff) | |
| download | colitur-7f263a0ec9a91d1a036cfd22ed38354d06500b1d.tar.gz colitur-7f263a0ec9a91d1a036cfd22ed38354d06500b1d.zip | |
fix(tools): harden check_citations.py against its own self-poisoning bug
Reproduced the defect: reintroducing the exact historical citation bug
(pointing class-1's citation back at LT.txt:12459, the value a prior
fix round corrected away from) made the tool report "147 citations
checked, 0 look wrong". The mechanism was that the corrective comment
documenting the old bug quotes the wrong historical value, and the
checker pooled every quoted phrase from the whole surrounding comment
block, so citing the wrong line matched the comment explaining why it
was wrong.
Four changes:
1. The word pool for a citation is now scoped to the entry(ies) it is
attached to only -- never to quoted text elsewhere in the comment.
This is the direct fix for the self-poisoning bug.
2. A citation whose pool has fewer than two distinctive words (Latin
liturgical headings are short and stopword-heavy) cannot
discriminate the right line from a wrong nearby one. Such a
citation is now reported CANNOT VERIFY and fails the target,
instead of silently passing.
3. The blanket +-2-line tolerance is gone. A bare "LT.txt:N" is
checked at line N only; a heading that genuinely wraps must say so
explicitly as "LT.txt:N-M". The allowance moves into the data,
where it is visible.
4. The tool gets its own test suite, tools/test_check_citations.py,
with a synthetic fixture covering: a correct citation, off-by-one
and off-by-three mismatches, an explicit wrap range, a degenerate
pool, a PATTERN-marked entry with no citation, and a dedicated
regression test for the self-poisoning case itself. Wired into
`dune test` via a new (rule (alias runtest) ...) in tools/dune (a
plain (test ...) stanza cannot run a Python script), so it runs
with the rest of the suite, not only as a `make` target.
Added a --file/--lt-file override to check_citations.py so the tool
(and its own tests) can point at a fixture without touching the real
lang/la.ini or docs/research/LT.txt. Confirmed the "SKIPPED, exit 0"
behaviour for a missing docs/research/LT.txt is unchanged.
tools/__pycache__/ (a stray artefact of this script, previously
untracked and ungitignored) is now in .gitignore.
Measured against the current lang/la.ini (another task is still
landing its sanctoral entries on this branch): 15 of 275 citations now
look wrong and 42 more cannot be verified, both far above the 0 the
unhardened tool reported. Not fixed here -- the data pass is separate,
once the sanctoral entries land.
Diffstat (limited to 'Makefile')
| -rw-r--r-- | Makefile | 23 |
1 files changed, 16 insertions, 7 deletions
@@ -88,13 +88,22 @@ check-templates: build ## typeset every shipped template (needs pdflatex/groff; # docs/research/LT.txt line it was transcribed from, so a claim can be # checked, not just trusted. tools/check_citations.py re-derives that check # mechanically: for every "LT.txt:<n>" citation outside a PATTERN-marked -# block, it confirms a +-2-line window around line n actually contains the -# Latin text the citation claims -- see that script's own docstring for the -# exact rule and its known limits (a heuristic, not a proof). docs/ is -# gitignored, so a fresh clone has no docs/research/LT.txt at all: the same -# "SKIPPED, loudly, exit 0" discipline check-schema/check-templates already -# use above -- a silent skip reads as a pass, which this project has hit -# the cost of before. +# block, it confirms the EXACT cited line (or an explicit "n-m" range, for +# a heading that genuinely wraps -- no automatic +-2-line tolerance any +# more, see the script's own docstring for why) contains the Latin text the +# citation claims, scoped to that citation's own entry only, never to +# prose quoted elsewhere in the surrounding comment. A citation whose pool +# of candidate words is too thin to discriminate a right line from a wrong +# nearby one (a heuristic, not a proof) is reported CANNOT VERIFY, not +# silently passed, and fails the target exactly like a genuine mismatch -- +# a human adjudicates it. The script has its own self-test +# (tools/test_check_citations.py, wired into `dune test`/`make test` as +# well as this target) after an earlier version of this exact check was +# found to be self-poisoning: see either docstring for the full account. +# docs/ is gitignored, so a fresh clone has no docs/research/LT.txt at +# all: the same "SKIPPED, loudly, exit 0" discipline check-schema/ +# check-templates already use above -- a silent skip reads as a pass, +# which this project has hit the cost of before. check-citations: ## verify lang/la.ini's LT.txt:<n> citations (needs docs/research/LT.txt, gitignored; SKIPPED if absent) @if command -v python3 >/dev/null 2>&1; then \ python3 tools/check_citations.py; \ |
