| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fold_ics's UTF-8 backoff loop could back `cut` all the way down to
`pos` on 74+ consecutive continuation bytes (0x80-0xBF), producing a
zero-length chunk and recursing on the identical position forever --
not producible by valid UTF-8, whose longest continuation run is 3,
but the kernel's own totality requirement covers arbitrary octet
strings, not only valid ones. When backoff finds no boundary inside
the window, cut hard at the limit instead, so forward progress is
unconditional.
test_fold_never_splits_utf8 previously asserted only that unfolding
reproduced the original bytes, a property folding preserves at any
cut position and therefore blind to a boundary violation. It now also
asserts the named property directly: no continuation chunk may start
with a UTF-8 continuation byte. A new regression test feeds fold_ics
100 consecutive continuation bytes and asserts it terminates with
every line at or under 75 octets.
|
|
|
Six flavours: latex, groff, html, xml, ics, none. Markdown, AsciiDoc and
plain text map to none deliberately -- their metacharacters are
context-dependent and escaping them aggressively produces worse output
than not escaping.
An unrecognised extension returns None rather than falling back to none:
guessing the flavour wrong produces malformed output that looks fine
until it does not.
Folding backs off to a non-continuation byte, so a fold never splits a
UTF-8 sequence -- the failure mode that would corrupt Polish and Latin
names in a published feed.
|
