| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds Escape.Typst: type, all, to_string ("typst"), of_string,
of_extension (.typ), and a per-character expand function, structured
identically to the existing latex escaper (one pass, no re-scan, so
double-escaping stays impossible by construction).
The metacharacter set was verified against the installed typst 0.14.2
binary, not assumed: a probe document escaping each of #, *, _, $, @,
<, >, `, \, ~ and - was compiled and pdftotext'd back to confirm the
literal character survives, and each was separately confirmed to do
something else when left bare (# opens code mode, */_ toggle
strong/emph, $ opens math, @ opens a reference -- a bare unresolved
@word is a hard compile error, not merely mangled output -- </> can
close around a bare word into label syntax that swallows it whole, `
opens raw, ~ is a non-breaking space, and a run of two or three '-'
becomes an en/em dash). All ten are backslash-escapable; none needed a
non-backslash workaround. '-' is escaped unconditionally rather than
only inside a detected run, since this escaper has no lookahead -- a
probe confirmed escaping every hyphen independently still typesets as
literal hyphens for a run of any length, so the single per-character
rule is sufficient.
test_escape.ml's new Typst cases were written first and shown to fail
against a stubbed identity apply before the real escaper landed, per
this project's own regression-test discipline. Comments in both files
mark which new tests are genuine regression tests (the per-character
escaping, including a real shipped citation and a synthetic
markdown-habit overlay name) versus characterisation (to_string/
of_string/of_extension are flat table lookups with no logic to have
been wrong).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fold_ics's UTF-8 backoff loop could back `cut` all the way down to
`pos` on 74+ consecutive continuation bytes (0x80-0xBF), producing a
zero-length chunk and recursing on the identical position forever --
not producible by valid UTF-8, whose longest continuation run is 3,
but the kernel's own totality requirement covers arbitrary octet
strings, not only valid ones. When backoff finds no boundary inside
the window, cut hard at the limit instead, so forward progress is
unconditional.
test_fold_never_splits_utf8 previously asserted only that unfolding
reproduced the original bytes, a property folding preserves at any
cut position and therefore blind to a boundary violation. It now also
asserts the named property directly: no continuation chunk may start
with a UTF-8 continuation byte. A new regression test feeds fold_ics
100 consecutive continuation bytes and asserts it terminates with
every line at or under 75 octets.
|
|
|
Six flavours: latex, groff, html, xml, ics, none. Markdown, AsciiDoc and
plain text map to none deliberately -- their metacharacters are
context-dependent and escaping them aggressively produces worse output
than not escaping.
An unrecognised extension returns None rather than falling back to none:
guessing the flavour wrong produces malformed output that looks fine
until it does not.
Folding backs off to a non-continuation byte, so a fold never splits a
UTF-8 sequence -- the failure mode that would corrupt Polish and Latin
names in a published feed.
|
