diff options
| author | Lukasz Kasprzak <lukas@labunix.xyz> | 2026-08-19 15:47:37 +0200 |
|---|---|---|
| committer | Lukasz Kasprzak <lukas@labunix.xyz> | 2026-08-19 15:47:37 +0200 |
| commit | 174fe8b3fedf61cf1fa0dc7499573374133a8ca2 (patch) | |
| tree | a2a5e7162b9e4392d9ffc272a41dbf1dce01223f /test/test_rite_ef.ml | |
| parent | 6d367ab8e90f6e713d262a7f19fb908b28d4796a (diff) | |
| download | colitur-174fe8b3fedf61cf1fa0dc7499573374133a8ca2.tar.gz colitur-174fe8b3fedf61cf1fa0dc7499573374133a8ca2.zip | |
fix(tools): close five more ways to defeat check_citations.py
Round 1 hardened check_citations.py against its own self-poisoning bug;
a review defeated it again. Five fixes, in the order they were found:
1. PATTERN silenced a whole comment block, not just the entry it was
attached to -- a wrong citation on a DIFFERENT, unmarked entry in the
same block (e.g. [season]'s own back-to-back trailing-comment style)
was never checked at all. Fixed by scoping PATTERN with the identical
leading/trailing pooling rule citations already use: an entry is
excluded only by its own marker, never a neighbour's.
2. Explicit per-citation ranges (introduced in round 1 to replace a
blanket +-2-line tolerance) had no upper bound, reintroducing the same
defect at a much larger radius (LT.txt:8600-8650 passed if the text
appeared anywhere in fifty lines). Capped at MAX_RANGE_WIDTH (3 lines);
anything wider is reported MALFORMED, naming the entry and the width,
instead of silently accepted.
3. The "pool too thin to verify" gate counted words, not rarity -- it
flagged 11 genuinely correct citations (short Latin hagionyms with only
one non-stopword) CANNOT VERIFY, while a match on nothing but "classis"
(507 occurrences) passed freely alongside three siblings. Replaced with
a frequency table over the whole LT.txt corpus: a token's evidence is
1/(times seen), an item's evidence is its single rarest matched token
(not a sum -- summing would let several merely-common words add up to
"enough" between them, the same shape as the self-poisoning bug).
4. "LT.txt:12,459" (a comma typo for one number) parsed as two unrelated
bare citations, 12 and 459, either of which could coincidentally match
while the intended line was never checked. Detected as a
thousands-separator-typo shape (a 1-2 digit token immediately followed
by an exactly-3-digit one -- the only way a real LT.txt line number,
which never exceeds 5 digits, splits under one comma) and rejected as
malformed.
5. The self-test suite overstated its own coverage: of round 1's seven
fixture cases, only two actually failed against the pre-round-1 script.
Every test is now labelled REGRESSION or CHARACTERISATION, each
verified by direct replay against the named prior version rather than
asserted -- 14 of 33 are genuine regression tests.
Both of the review's own defeats (block-wide PATTERN silencing, the
50-line range) are reproduced as dedicated fixtures and confirmed caught;
both are also confirmed to slip through the pre-round-2 tool unchanged.
Claude-Session: https://claude.ai/code/session_017ZBxCCRM2ojnBupp3SBxV9
Diffstat (limited to 'test/test_rite_ef.ml')
0 files changed, 0 insertions, 0 deletions
