From decb13fb17697f6d6d952c407f2173a714164804 Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 19 Aug 2026 09:19:14 +0200 Subject: feat(cli): colitur emit -- csv, json, sexp, xml, ics Reuses resolved_year_report's existing two-liturgical-year indexing rather than copying it: that walk owns the civil-vs-liturgical span reasoning, and a second copy would drift. It is refactored to return the days, with the printer layered on top, so day and readings behave identically -- which cli.t proves byte-for-byte. CSV emits one header for a whole multi-year run, not one per year. A reversed range is a usage error rather than silently empty output. Asserted in cli.t: two ics runs are byte-identical, because nothing in the path reads a clock. --- test/cli.t | 69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) (limited to 'test/cli.t') diff --git a/test/cli.t b/test/cli.t index 1d4df3a..9b3373a 100644 --- a/test/cli.t +++ b/test/cli.t @@ -415,3 +415,72 @@ displaced silently. $ colitur day 2026 --overlay ben.sexp | grep '^2026-03-21' 2026-03-21 saturday lent 4 transitus-of-our-holy-father-benedict class-1 white +ef-lent-4-saturday + +CSV emits a header and one row per day: + + $ colitur emit --format csv --from 2027 --to 2027 | head -2 + date,rite,season,week,slug,rank,colour,subject,name_la,name_en,first,gospel,comms + 2027-01-01,ef,christmastide,,ef-circumcision,class-1,white,temporal,,,Titus 2:11-15,Luke 2:21, + + $ colitur emit --format csv --from 2027 --to 2027 | wc -l + 366 + +JSON is one object, ICS one VCALENDAR: + + $ colitur emit --format json --from 2027 --to 2027 | cut -c1-20 + {"rite":"ef","year": + + $ colitur emit --format ics --from 2027 --to 2027 | head -1 | cat -A | head -1 + BEGIN:VCALENDAR^M$ + +Two runs are byte-identical (no clock read anywhere): + + $ colitur emit --format ics --from 2027 --to 2027 > /tmp/a.ics + $ colitur emit --format ics --from 2027 --to 2027 > /tmp/b.ics + $ cmp /tmp/a.ics /tmp/b.ics && echo identical + identical + +A multi-year range concatenates years in order, one header for the whole +CSV run rather than one per year: + + $ colitur emit --format csv --from 2027 --to 2028 | grep -c '^2028-' + 366 + + $ colitur emit --format csv --from 2027 --to 2028 | wc -l + 732 + +sexp and xml are also available: + + $ colitur emit --format sexp --from 2027 --to 2027 | wc -l + 8472 + + $ colitur emit --format xml --from 2027 --to 2027 | head -2 + + + +An unknown format is a usage error on stderr, exit 2: + + $ colitur emit --format yaml --from 2027 --to 2027 + colitur: unknown format "yaml" (want csv, json, sexp, xml or ics) + [2] + +emit refuses a reversed range rather than emitting nothing: + + $ colitur emit --format csv --from 2028 --to 2027 + colitur: --from 2028 is after --to 2027 + [2] + +emit's own flags have no effect on the other commands, refused rather than +silently ignored, the same discipline --overlay already gets: + + $ colitur day 2027 --format csv + colitur: --format/--from/--to/--dtstamp have no effect on `day`; refusing rather than ignoring them + [2] + +day and readings are untouched: + + $ colitur day 2027 | head -1 + 2027-01-01 friday christmastide - ef-circumcision class-1 white + + $ colitur readings 2027 | head -1 + 2027-01-01 ef-circumcision | Titus 2:11-15 | Luke 2:21 -- cgit v1.3 From 99172c86c518fdb2104c098b7b7a79e6c13ba8ea Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 19 Aug 2026 09:35:52 +0200 Subject: feat(cli): colitur table and render Computes and renders in one process. There is deliberately no stdin-fed render: honouring the pipe would need a JSON parser we would have to write, purely to serialise and immediately re-parse our own view -- a second hand-rolled component and a second place for the contract to drift, for no benefit. colitur emit --format json | jq still composes. An unknown extension with no --flavour is an error naming the six valid flavours, never a silent fallback to none: guessing wrong produces malformed output that looks fine until it does not. A malformed template reports the parser's own reason and exits 2. A template is user input; it must never crash the program. --- bin/main.ml | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- man/colitur.1 | 183 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++- test/cli.t | 69 ++++++++++++++++++++++ 3 files changed, 428 insertions(+), 3 deletions(-) (limited to 'test/cli.t') diff --git a/bin/main.ml b/bin/main.ml index ec7fd71..5e3d07d 100644 --- a/bin/main.ml +++ b/bin/main.ml @@ -426,6 +426,75 @@ let emit_report ~format ~overlays ~dtstamp ~from_y ~to_y = exit 2 done +(* Task 9: `colitur table` and `colitur render` -- compute a year and render it + through a user-supplied template, in ONE process. + + The design's own sketch was `compute | render` as a Unix pipe, with `render` + reading a serialised view back from stdin. That is deliberately NOT built: + honouring the pipe would need a JSON *parser*, purely to re-read the view + this same process just serialised -- a second hand-rolled component, and a + second place for the published contract to drift, for no benefit over + calling [View.of_days] directly. So `table --year Y --template F` computes + and renders in one process (the command that actually gets used), and + `render --template F --year Y` is the identical operation under the name + the design used, kept so that documented vocabulary still works. There is + no stdin-fed `render`; `colitur emit --format json | jq` still composes for + real pipe use, because JSON there is the OUTPUT, never something colitur + itself has to parse back in. *) + +let read_file path = + match open_in_bin path with + | exception Sys_error _ -> Error ("cannot read template " ^ path) + | ic -> + let n = in_channel_length ic in + let s = really_input_string ic n in + close_in ic; + Ok s + +let extension path = + match String.rindex_opt path '.' with + | Some i -> String.sub path i (String.length path - i) + | None -> "" + +(* An unknown extension with no [--flavour] is an ERROR, never a silent + fallback to [Escape.None_]: guessing the flavour wrong produces malformed + output (unescaped LaTeX/HTML metacharacters) that looks fine until it does + not -- the same "never silently substitute" discipline [data_dir]'s own + [COLITUR_DATA_DIR] handling documents above. *) +let table_report ~template ~flavour_opt ~overlays y = + let flavour = + match flavour_opt with + | Some name -> ( + match Colitur_render.Escape.of_string name with + | Some f -> f + | None -> + Printf.eprintf "colitur: unknown flavour %S (want latex, groff, html, xml, ics or none)\n" + name; + exit 2) + | None -> ( + match Colitur_render.Escape.of_extension (extension template) with + | Some f -> f + | None -> + Printf.eprintf + "colitur: cannot infer a flavour from %S; pass --flavour latex|groff|html|xml|ics|none\n" + (extension template); + exit 2) + in + match read_file template with + | Error msg -> + Printf.eprintf "colitur: %s\n" msg; + exit 2 + | Ok src -> ( + let days = resolved_year_days ~overlays y in + let v = Colitur_render.View.of_days ~vocab:Rite_ef.Vocab_ef.vocab ~rite:"ef" ~year:y days in + match Colitur_render.Template.render_string ~flavour src v with + | Error e -> + (* The template is user input; a parse failure is reported with the + parser's OWN reason and exits 2, never an uncaught exception. *) + Printf.eprintf "colitur: template %s: %s\n" template e; + exit 2 + | Ok out -> print_string out) + (* Help and usage are deliberately DIFFERENT things, and the difference is the Unix convention rather than a preference: asking for help is a request that SUCCEEDED, so [--help] prints to stdout and exits 0 (it can be piped into a @@ -455,6 +524,11 @@ usage: colitur emit --format csv|json|sexp|xml|ics --from Y --to Y [--overlay FILE ...] [--dtstamp S] render a resolved year range through one of five emitters + colitur table --year Y --template FILE [--flavour X] [--overlay FILE ...] + colitur render --template FILE --year Y [--flavour X] [--overlay FILE ...] + compute year Y and render it through FILE, a logic-less + Mustache-family template; table and render are the same + operation, two names (see "rendering" below) colitur new-overlay print a starter overlay file to stdout colitur convert FILE.ini flat INI overlay -> S-expression, on stdout colitur check FILE ... load an overlay, say what it does, exit 2 if not @@ -517,6 +591,55 @@ overlays: (Nth_weekday (month M) (nth N) (weekday W)) with N negative to count from the end of the month. +rendering: + --template FILE (required on `table`/`render`) is a logic-less Mustache- + family template: {{placeholder}}, {{#section}}...{{/section}}, + {{^inverted}}...{{/inverted}}, {{!comment}} -- nothing else. It is + DATA, never a program: no partials, no lambdas, no expression + evaluation, no filesystem or process access, and no "raw" or + triple-brace form that could opt out of escaping. The value it + renders against is the same schema `emit` uses (season, week, + slug, rank, colour, subject, names, citations, commemorations), + reshaped into a booklet (`days`) and a month grid (`weeks`, with + padding cells for the leading/trailing blanks); see colitur(1) + for the full field list. + + --flavour X selects how interpolated VALUES are escaped (never the + template's own literal markup, which is the author's). One of: + + latex groff html xml ics none + + Inferred from --template's extension when --flavour is omitted: + + .tex -> latex + .ms .mom .me -> groff + .html .htm -> html + .xml -> xml + .ics -> ics + .md .adoc .txt -> none (no metacharacters are escaped; + Markdown/AsciiDoc/plain text have no fixed + metacharacter set, so escaping them here + would produce worse output than leaving + them alone) + + An extension colitur does not recognise is a hard ERROR naming + the six flavours above, never a silent fallback to `none`: + guessing wrong produces output that looks fine until the + metacharacters it silently failed to escape show up. + + `table` and `render` are the SAME operation under two names. The design + this project followed originally sketched `compute | render` as a + Unix pipe, with `render` reading a serialised view back from + stdin. That is deliberately not built: honouring the pipe would + need a JSON *parser*, purely so this program could re-read a view + it had just serialised itself -- a second hand-rolled component, + and a second place for the published contract to drift, for no + benefit over calling the view builder directly in the same + process. There is therefore no stdin-fed `render`; `colitur emit + --format json | jq` still composes for real pipe use, because + that JSON is the OUTPUT, never something colitur itself parses + back in. + environment: COLITUR_DATA_DIR Read the calendar data from this directory instead of the @@ -564,12 +687,19 @@ let with_year ys f = [--format]/[--from]/[--to]/[--dtstamp] (Task 8, `emit`) are each single- valued, unlike [--overlay], so they are plain [string option] fields rather than accumulating lists. *) +(* [year]/[template]/[flavour] (Task 9, `table`/`render`) are each single- + valued, the same shape as [format]/[from_y]/[to_y]/[dtstamp] above -- + `table`/`render` take one year and one template file, never a range or a + repeatable list. *) type parsed_args = { overlays : string list; format : string option; from_y : string option; to_y : string option; dtstamp : string option; + year : string option; + template : string option; + flavour : string option; positional : string list; } @@ -586,6 +716,12 @@ let parse_args argv = | [ "--to" ] -> Error "--to needs a value" | "--dtstamp" :: v :: rest -> go { acc with dtstamp = Some v } rest | [ "--dtstamp" ] -> Error "--dtstamp needs a value" + | "--year" :: v :: rest -> go { acc with year = Some v } rest + | [ "--year" ] -> Error "--year needs a value" + | "--template" :: v :: rest -> go { acc with template = Some v } rest + | [ "--template" ] -> Error "--template needs a value" + | "--flavour" :: v :: rest -> go { acc with flavour = Some v } rest + | [ "--flavour" ] -> Error "--flavour needs a value" (* The recognised bare flags pass through as positional words for the dispatch below to match; anything else beginning with '-' is rejected rather than silently treated as a command or a year. *) @@ -596,7 +732,10 @@ let parse_args argv = Error (Printf.sprintf "unknown option %s" arg) | arg :: rest -> go { acc with positional = arg :: acc.positional } rest in - go { overlays = []; format = None; from_y = None; to_y = None; dtstamp = None; positional = [] } argv + go + { overlays = []; format = None; from_y = None; to_y = None; dtstamp = None; year = None; + template = None; flavour = None; positional = [] } + argv (* Sibling to [reject_overlays_for]: `emit`'s own four flags have no meaning on any other command (they take a single [] positional, not a @@ -620,6 +759,18 @@ let reject_overlays_for cmd overlays = exit 2 end +(* Sibling to [reject_emit_flags_for]/[reject_overlays_for]: `table`/`render`'s + own three flags (Task 9) have no meaning on any other command, so accepting + and silently dropping them would be the same failure mode this project + already refuses everywhere else. *) +let reject_table_flags_for cmd ~year ~template ~flavour = + if year <> None || template <> None || flavour <> None then begin + Printf.eprintf + "colitur: --year/--template/--flavour have no effect on `%s`; refusing rather than ignoring them\n" + cmd; + exit 2 + end + (* `colitur check FILE...` -- load a user overlay, apply it to the real shipped calendar, and say what it did, without printing a year of output. @@ -784,46 +935,57 @@ let () = | Error msg -> Printf.eprintf "colitur: %s\n" msg; usage () - | Ok { overlays; format; from_y; to_y; dtstamp; positional } -> ( + | Ok { overlays; format; from_y; to_y; dtstamp; year; template; flavour; positional } -> ( let reject_emit = reject_emit_flags_for ~format ~from_y ~to_y ~dtstamp in + let reject_table = reject_table_flags_for ~year ~template ~flavour in match positional with | [ ("-h" | "--help" | "help") ] -> reject_overlays_for "--help" overlays; reject_emit "--help"; + reject_table "--help"; print_help () | [ ("-V" | "--version" | "version") ] -> reject_overlays_for "--version" overlays; reject_emit "--version"; + reject_table "--version"; print_endline version; exit 0 | [ "easter"; ys ] -> reject_overlays_for "easter" overlays; reject_emit "easter"; + reject_table "easter"; with_year ys easter_report | [ "temporal"; ys ] -> reject_overlays_for "temporal" overlays; reject_emit "temporal"; + reject_table "temporal"; with_year ys temporal_report | "check" :: (_ :: _ as files) -> reject_overlays_for "check" overlays; reject_emit "check"; + reject_table "check"; check_report files | [ "convert"; path ] -> reject_overlays_for "convert" overlays; reject_emit "convert"; + reject_table "convert"; convert_report path | [ "new-overlay" ] -> reject_overlays_for "new-overlay" overlays; reject_emit "new-overlay"; + reject_table "new-overlay"; print_string new_overlay_template; exit 0 | [ "day"; ys ] -> reject_emit "day"; + reject_table "day"; with_year ys (day_report ~overlays) | [ "readings"; ys ] -> reject_emit "readings"; + reject_table "readings"; with_year ys (readings_report ~overlays) | [ "emit" ] -> ( + reject_table "emit"; match format with | None -> Printf.eprintf "colitur: emit requires --format csv|json|sexp|xml|ics\n"; @@ -836,4 +998,17 @@ let () = | Some from_ys, Some to_ys -> with_year from_ys (fun from_y -> with_year to_ys (fun to_y -> emit_report ~format ~overlays ~dtstamp ~from_y ~to_y)))) + | [ ("table" | "render") as cmd ] -> ( + reject_emit cmd; + match template with + | None -> + Printf.eprintf "colitur: %s requires --year YEAR and --template FILE\n" cmd; + exit 2 + | Some template -> ( + match year with + | None -> + Printf.eprintf "colitur: %s requires --year YEAR and --template FILE\n" cmd; + exit 2 + | Some ys -> with_year ys (fun y -> table_report ~template ~flavour_opt:flavour ~overlays y) + )) | _ -> usage ()) diff --git a/man/colitur.1 b/man/colitur.1 index 1db7f9a..da08a36 100644 --- a/man/colitur.1 +++ b/man/colitur.1 @@ -21,6 +21,13 @@ colitur \- deterministic liturgical calendar and lectionary engine (Roman rite, .RB [ \-\-dtstamp " STAMP" ] .br .B colitur +.BR table | render +.BI \-\-year " YEAR" +.BI \-\-template " FILE" +.RB [ \-\-flavour " FLAVOUR" ] +.RB [ \-\-overlay " FILE" " ...]" +.br +.B colitur .BR \-h | \-\-help .SH DESCRIPTION .B colitur @@ -80,6 +87,17 @@ See .B EMIT below. .TP +.BR table | render +Compute one civil year and render it through a user\-supplied template, in one +process. +.B table +and +.B render +are the same operation under two names \(em see +.B RENDERING +below for why there is no separate, stdin\-fed +.B render . +.TP .BI convert " FILE" .ini Convert a flat INI overlay to the S\-expression form, on standard output. The conversion verifies its own output before emitting it: the generated text is @@ -112,7 +130,7 @@ below. .TP .BI \-\-overlay " FILE" Apply a user calendar on top of the shipped one. Repeatable and ordered; -.BR day ", " readings " and " emit +.BR day ", " readings ", " emit ", " table " and " render only. See .B OVERLAYS below. @@ -145,6 +163,28 @@ is the emitted year. Never a clock read either way \(em see .B EMIT below. .TP +.BI \-\-year " YEAR" +.RB ( "colitur table" " and " "colitur render" " only)" +The civil year to compute, +.B 1583..9999 +as elsewhere. Required. +.TP +.BI \-\-template " FILE" +.RB ( "colitur table" " and " "colitur render" " only)" +The template file to render the year through. Required. See +.B RENDERING +below. +.TP +.BI \-\-flavour " FLAVOUR" +.RB ( "colitur table" " and " "colitur render" " only)" +One of +.BR latex ", " groff ", " html ", " xml ", " ics " or " none . +Overrides the flavour that would otherwise be inferred from +.BR \-\-template 's +own extension. See +.B RENDERING +below. +.TP .BR \-h ", " \-\-help Print a usage summary to standard output and exit 0. .TP @@ -299,6 +339,139 @@ applied on top of the shipped calendar, in order, before the range is rendered. See .B OVERLAYS below. +.SH RENDERING +.BI "colitur table " \-\-year " YEAR " \-\-template " FILE" +and +.BI "colitur render " \-\-template " FILE " \-\-year " YEAR" +are the +.I same +operation under two names: compute the resolved year, shape it into the +same view +.B emit +uses, and render it through +.I FILE +in one process. Both accept +.BR \-\-flavour " and " \-\-overlay +identically. +.SS Why there is no stdin\-fed render +The design this project followed originally sketched a Unix pipe, +.BR "compute | render" , +with +.B render +reading a serialised view back from standard input. That is deliberately +.I not +built. +Honouring the pipe would require a JSON +.I parser +inside +.B colitur +\(em a second hand\-rolled component, purely so this program could read back a +view it had just serialised itself, and a second place for the published +output schema to drift out of step with what the parser actually accepts. +That is a real cost for no benefit over calling the same view builder +directly in the same process, which is what +.B table +and +.B render +both do. +.PP +Unix composition is not abandoned, only narrowed to where it is cheap and +honest: +.B "colitur emit \-\-format json | jq" +still composes fine, because that JSON is the +.I output +of the pipeline, never something +.B colitur +itself has to parse back in. +.SS Templates +.I FILE +is a deliberately logic\-less, Mustache\-family template: it is +.I data, +never a program. The only constructs are +.BR {{placeholder}} , +.BR {{#section}}...{{/section}} , +.BR {{^inverted}}...{{/inverted}} +and +.BR {{!comment}} . +There are no partials, no lambdas, no expression evaluation, no arithmetic, +and no filesystem or process access from inside a template. There is +deliberately no "raw" or triple\-brace form either \(em a template cannot opt +out of its flavour's escaping. +.PP +The template renders against the same schema +.B emit +uses (season, week, slug, rank, colour, subject, names, citations, +commemorations), reshaped for two artefacts from one model: a flat booklet +(the +.B days +list, one entry per day of the year) and a month grid (the +.B weeks +list, with padding cells flagged for the leading and trailing blanks a grid +needs and a booklet does not). A key absent on a given day (an optional field +a rite does not always set) renders as the empty string rather than an error +\(em the one deliberate silence, so a template survives a day that does not +carry every optional field. +.SS Flavours +.BI \-\-flavour +controls how interpolated +.I values +are escaped for the target format. It never touches the template's own +literal markup, which is the author's and is trusted as\-is. One of: +.RS +.nf + +latex groff html xml ics none +.fi +.RE +.PP +When +.B \-\-flavour +is omitted it is inferred from +.BR \-\-template 's +own file extension: +.RS +.nf + +.I .tex -> latex +.I .ms .mom .me -> groff +.I .html .htm -> html +.I .xml -> xml +.I .ics -> ics +.I .md .adoc .txt -> none +.fi +.RE +.PP +.B none +escapes nothing: Markdown, AsciiDoc and plain text have no fixed +metacharacter set, so escaping them here would produce worse output than +leaving them alone. +.PP +An extension +.B colitur +does not recognise is a hard error naming the six flavours above; it is +.I never +a silent fallback to +.BR none . +Guessing the flavour wrong produces output that looks fine right up until +the metacharacters it silently failed to escape show up in a rendered +document. +.RS +.nf + +.B colitur table \-\-year 2027 \-\-template invite.wat +colitur: cannot infer a flavour from ".wat"; pass \-\-flavour latex|groff|html|xml|ics|none +.fi +.RE +.PP +A malformed template reports the parser's own reason and exits 2, never a +crash \(em a template is user input, exactly like an overlay file. +.RS +.nf + +.B colitur table \-\-year 2027 \-\-template bad.txt +colitur: template bad.txt: unclosed section {{#days}} +.fi +.RE .SH OVERLAYS .TP .BI \-\-overlay " FILE" @@ -484,6 +657,14 @@ Run against a checkout's data rather than the installed copy: .B COLITUR_DATA_DIR=~/git/projects/colitur/data/ef colitur day 2026 .fi .RE +.PP +Render a year through a template, flavour inferred from the extension: +.RS +.nf + +.B colitur table \-\-year 2026 \-\-template booklet.tex > booklet.tex.out +.fi +.RE .SH SOURCES The calendar is computed against the 1962 .I Missale Romanum diff --git a/test/cli.t b/test/cli.t index 9b3373a..73eefc1 100644 --- a/test/cli.t +++ b/test/cli.t @@ -484,3 +484,72 @@ day and readings are untouched: $ colitur readings 2027 | head -1 2027-01-01 ef-circumcision | Titus 2:11-15 | Luke 2:21 + +A minimal inline template renders -- table computes and renders in one +process (2 January 2027 is a Saturday, not a Sunday, so Holy Name Sunday +falls on the 3rd, not the 2nd, that year): + + $ printf '{{#days}}{{iso}} {{slug}}\n{{/days}}' > /tmp/t.txt + $ colitur table --year 2027 --template /tmp/t.txt | head -2 + 2027-01-01 ef-circumcision + 2027-01-02 ef-christmas-1-saturday + +render is the same operation under the name the design used: + + $ colitur render --template /tmp/t.txt --year 2027 | head -2 + 2027-01-01 ef-circumcision + 2027-01-02 ef-christmas-1-saturday + +Flavour is inferred from the extension and escapes data -- Sts. Peter & +Paul (29 June) and its vigil are the only two 2035 entries whose English +name needs LaTeX escaping: + + $ printf '{{#days}}{{name.en}}\n{{/days}}' > /tmp/t.tex + $ colitur table --year 2035 --template /tmp/t.tex | grep -c 'Peter \\& Paul' + 2 + +An unknown extension with no --flavour is an error, not a silent fallback: + + $ printf 'x' > /tmp/t.wat + $ colitur table --year 2027 --template /tmp/t.wat + colitur: cannot infer a flavour from ".wat"; pass --flavour latex|groff|html|xml|ics|none + [2] + + $ colitur table --year 2027 --template /tmp/t.wat --flavour none + x + +An unrecognised --flavour value is also an error naming the six valid ones: + + $ colitur table --year 2027 --template /tmp/t.txt --flavour bogus + colitur: unknown flavour "bogus" (want latex, groff, html, xml, ics or none) + [2] + +A malformed template is a clear error, not a crash: + + $ printf '{{#days}}oops' > /tmp/bad.txt + $ colitur table --year 2027 --template /tmp/bad.txt + colitur: template /tmp/bad.txt: unclosed section {{#days}} + [2] + +A missing template file is an error: + + $ colitur table --year 2027 --template /tmp/nope.txt + colitur: cannot read template /tmp/nope.txt + [2] + +table and render both require --year and --template: + + $ colitur table --year 2027 + colitur: table requires --year YEAR and --template FILE + [2] + + $ colitur render --template /tmp/t.txt + colitur: render requires --year YEAR and --template FILE + [2] + +table/render's own flags have no effect on the other commands, refused +rather than silently ignored: + + $ colitur emit --format csv --from 2027 --to 2027 --year 2028 + colitur: --year/--template/--flavour have no effect on `emit`; refusing rather than ignoring them + [2] -- cgit v1.3 From 6bd741bd512904dfec7c1aa2b7b2bd3dd4269681 Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 19 Aug 2026 09:53:37 +0200 Subject: fix(cli): guard the whole template read, not only the open read_file guarded open_in_bin but left in_channel_length and really_input_string unguarded, so a path that opens but cannot be read as bytes -- a directory -- escaped as an uncaught Sys_error and crashed the program, leaking the open channel on every failure path. A template is user input; it must never crash the program. Wrap the whole read in Fun.protect so the channel closes on every path (success, exception, early return), matching the close-on-every-path pattern already used in the test suite. The missing-file message stays exactly as before; a read failure after a successful open now carries the exception text, the same path: exception shape Layer.load and Overlay.load already use. New cram case points --template at a directory (the sandbox's own cwd, not /tmp) and asserts one stderr line and exit 2, not a crash. --- bin/main.ml | 31 ++++++++++++++++++++++++++----- test/cli.t | 9 +++++++++ 2 files changed, 35 insertions(+), 5 deletions(-) (limited to 'test/cli.t') diff --git a/bin/main.ml b/bin/main.ml index 5e3d07d..e916cbe 100644 --- a/bin/main.ml +++ b/bin/main.ml @@ -442,14 +442,35 @@ let emit_report ~format ~overlays ~dtstamp ~from_y ~to_y = real pipe use, because JSON there is the OUTPUT, never something colitur itself has to parse back in. *) +(* The open is guarded separately from the read: a missing file fails at + [open_in_bin] with a plain, path-only message (matching the wording this + project already uses for every other "no such file" case), while a file + that opens but cannot be READ -- a directory, a device node, anything + whose length or content changes between [open] and [read] -- fails inside + the [Fun.protect]'d body instead, carrying the raised exception's own text + (mirrors {!Colitur_kernel.Layer.load}/{!Colitur_kernel.Overlay.load}'s own + catch-all shape, lib/kernel/layer.ml and lib/kernel/overlay.ml). Either + way the channel is closed on EVERY path -- success, exception, or an + early return -- because [close_in_noerr] runs in [~finally], which + [Fun.protect] guarantees runs even when the protected function raises; a + bare [close_in] after [really_input_string] only ever ran on the success + path, leaking the descriptor on every failure. The whole read is inside + the [try], not only [open_in_bin], because [in_channel_length] and + [really_input_string] can themselves raise [Sys_error] (a directory opens + fine but is not readable as bytes) -- a template is user input, and this + project's own rule is that user input must never crash the program. *) let read_file path = match open_in_bin path with | exception Sys_error _ -> Error ("cannot read template " ^ path) - | ic -> - let n = in_channel_length ic in - let s = really_input_string ic n in - close_in ic; - Ok s + | ic -> ( + try + Fun.protect + ~finally:(fun () -> close_in_noerr ic) + (fun () -> + let n = in_channel_length ic in + let s = really_input_string ic n in + Ok s) + with exn -> Error (Printf.sprintf "cannot read template %s: %s" path (Printexc.to_string exn))) let extension path = match String.rindex_opt path '.' with diff --git a/test/cli.t b/test/cli.t index 73eefc1..c5aeb98 100644 --- a/test/cli.t +++ b/test/cli.t @@ -537,6 +537,15 @@ A missing template file is an error: colitur: cannot read template /tmp/nope.txt [2] +Pointing --template at a directory is an error, not a crash: the read +itself is guarded, not only the open (F1, fix round 1). "." is used rather +than a fixed /tmp path so this does not depend on anything outside the +cram sandbox itself: + + $ colitur table --year 2027 --template . --flavour none + colitur: cannot read template .: Sys_error("Value too large for defined data type") + [2] + table and render both require --year and --template: $ colitur table --year 2027 -- cgit v1.3 From 2760d43d695ba08fc33f65357590675707b6570d Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 19 Aug 2026 10:18:24 +0200 Subject: feat(cli): colitur publish -- the static tree Writes ef/.{json,csv,xml,ics}, one JSON per day, the schema and a generated index. That tree is the API: any web server or git repo serves it, and nothing runs at request time. Deterministic: publishing twice is byte-identical, asserted in cli.t. That is what makes publishing into a git repo safe -- the diff shows only real change, and you review it before pushing. Non-destructive: a manifest records exactly the files this tool wrote, so --prune can only remove files a previous run created. A file you put in the output directory yourself is never touched, with or without --prune. Asserted in both directions. Pruning a stale file also removes any directory it leaves empty behind it (e.g. an old year's own ef// tree), stopping at --out itself -- without this, a pruned year's own directory would survive empty and test -d would still see it. schema/day-v1.json is resolved the same prefix-relative way data/ef's own sexp files are (installed vs build-tree, probed rather than assumed), never from cwd, and a missing schema fails with one line on stderr before anything is written rather than emitting an empty file. Needed schema/day-v1.json wired into the root dune file's default alias and into test/dune's cram deps -- unlike data/ and templates/, nothing made dune mirror schema/ into the build tree before this. unix is added to bin/dune's libraries for mkdir_p; it ships with the compiler, so colitur.opam and dune-project are unchanged. --- bin/dune | 6 +- bin/main.ml | 269 ++++++++++++++++++++++++++++++++++- dune | 14 +- man/colitur.1 | 157 ++++++++++++++++++++- test/cli.t | 442 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ test/dune | 8 +- 6 files changed, 888 insertions(+), 8 deletions(-) (limited to 'test/cli.t') diff --git a/bin/dune b/bin/dune index 376fe70..856dafe 100644 --- a/bin/dune +++ b/bin/dune @@ -2,4 +2,8 @@ (name main) (public_name colitur) (package colitur) - (libraries colitur_kernel rite_ef colitur_render)) + ; [unix] ships with the OCaml compiler -- it is not a new entry in + ; colitur.opam's frozen depends, only a new library this executable links + ; against. Used by Task 12's [mkdir_p] (colitur publish, recursive + ; directory creation) and nowhere else. + (libraries colitur_kernel rite_ef colitur_render unix)) diff --git a/bin/main.ml b/bin/main.ml index e916cbe..1669a47 100644 --- a/bin/main.ml +++ b/bin/main.ml @@ -516,6 +516,183 @@ let table_report ~template ~flavour_opt ~overlays y = exit 2 | Ok out -> print_string out) +(* Task 12: `colitur publish` -- writes the static tree that IS this + project's API: a set of files any web server or git repo can serve as-is, + computed once, with nothing running at request time. + + Two properties matter more than anything else here: + + DETERMINISTIC -- publishing the same year range twice must produce a + byte-identical tree. That is what makes publishing into a git repo safe: + `git status` shows only genuine change, and a human reviews a real diff + before pushing. Nothing below reads a wall clock; [dtstamp] is threaded + through as a plain parameter all the way to + {!Colitur_render.Emit_ics.year}, exactly as `emit --format ics` already + requires (see that command's own comment above). + + NON-DESTRUCTIVE -- publish writes only files it owns, names every one of + them in a manifest ([.colitur-manifest], one relative path per line, + itself never subject to pruning), and [--prune] removes only entries + THAT MANIFEST lists which this run did not rewrite. A file the caller put + in the output directory themselves is never in the manifest, so it is + never touched, with or without [--prune] -- asserted in both directions + in test/cli.t. *) + +let rec mkdir_p path = + if path <> "" && path <> "/" && not (Sys.file_exists path) then begin + mkdir_p (Filename.dirname path); + try Unix.mkdir path 0o755 with Unix.Unix_error (Unix.EEXIST, _, _) -> () + end + +let write_file path contents = + mkdir_p (Filename.dirname path); + let oc = open_out_bin path in + output_string oc contents; + close_out oc + +let manifest_name = ".colitur-manifest" + +(* [read_file] rather than a second hand-rolled reader -- see its own + comment above for why the whole read, not only the open, is guarded. A + missing manifest (the very first publish into a fresh directory) is not + an error here: it just means there is nothing yet to prune against. *) +let read_manifest out = + match read_file (Filename.concat out manifest_name) with + | Error _ -> [] + | Ok contents -> String.split_on_char '\n' contents |> List.filter (fun l -> l <> "") + +(* [--prune] deletes the FILES a stale manifest entry names, but that alone + can leave their parent directories (ef///, then ef//) + empty behind them -- and an empty directory still makes `test -d + out/ef/` true, which is exactly the check a caller uses to confirm + an old year is gone. Walk upward from each deleted file's own directory, + removing it while it is empty, stopping at (never including) [out] + itself: [out] is the caller's own directory, never ours to remove, even + when it is empty. *) +let rec prune_empty_dirs ~out dir = + if dir <> out && String.length dir > String.length out && Sys.file_exists dir then + match Sys.readdir dir with + | [||] -> + (try Unix.rmdir dir with Unix.Unix_error _ -> ()); + prune_empty_dirs ~out (Filename.dirname dir) + | _ -> () + | exception Sys_error _ -> () + +(* schema/day-v1.json is resolved the same prefix-relative way [data_dir] + above resolves data/ef/*.sexp -- NOT from the process's own cwd, which + would break an installed binary invoked from an arbitrary directory. Two + candidates, installed then build-tree, the same shape as [data_dir]; a + candidate counts only if the file is actually there. No COLITUR_DATA_DIR + override here: that variable's whole contract is about the directory + holding sanctoral.sexp, and schema/ is not nested inside it. + + The installed candidate assumes schema/ lands at + /share/colitur/schema/day-v1.json, mirroring data/dune's own ef/ + layout. Adding that install rule is explicitly Task 13's job, not this + one -- this function only has to be ready to find the file once the rule + exists, which is why it is PROBED rather than assumed, exactly like + [data_dir]'s own installed candidate. *) +let schema_path () = + let prefix = Filename.dirname (Filename.dirname Sys.executable_name) in + let installed = + List.fold_left Filename.concat prefix [ "share"; "colitur"; "schema"; "day-v1.json" ] + in + let build_tree = List.fold_left Filename.concat prefix [ "schema"; "day-v1.json" ] in + if Sys.file_exists installed then Some installed else if Sys.file_exists build_tree then Some build_tree else None + +(* An ordinary OCaml string, NOT a template: it describes the TREE, not the + calendar, so it has no business in the template vocabulary. *) +let index_html ~from_y ~to_y = + let b = Buffer.create 4096 in + Buffer.add_string b + "\n\n\ + colitur\n\ + \n\ +

colitur

\n\ +

Liturgical calendar of the 1962 Missale Romanum. Citations only \xe2\x80\x94 never scripture text.

\n"; + Buffer.add_string b "

Subscribe

\n
    \n"; + for y = from_y to to_y do + Buffer.add_string b (Printf.sprintf "
  • ef/%d.ics
  • \n" y y) + done; + Buffer.add_string b "
\n

Data

\n
    \n"; + for y = from_y to to_y do + Buffer.add_string b + (Printf.sprintf + "
  • %d: json csv \ + xml \xe2\x80\x94 per-day at ef/%d/MM/DD.json
  • \n" + y y y y y) + done; + Buffer.add_string b + "
\n

Contract: schema/day-v1.json

\n\ + \n"; + Buffer.contents b + +let publish_report ~from_y ~to_y ~out ~overlays ~dtstamp ~prune = + if from_y > to_y then begin + Printf.eprintf "colitur: --from %d is after --to %d\n" from_y to_y; + exit 2 + end; + (* Resolved and read BEFORE any file is written, so a missing/unreadable + schema fails fast, before the output directory has anything half- + written in it. [read_file]'s own error text says "cannot read + template ..." (it was built for Task 9's template reads) -- accurate + about the mechanism, wrong about the noun, so the message here is + rebuilt rather than printed verbatim. *) + let schema = + match schema_path () with + | None -> + Printf.eprintf + "colitur: cannot find schema/day-v1.json (looked in the installed and build-tree locations)\n"; + exit 2 + | Some p -> ( + match read_file p with + | Error _ -> + Printf.eprintf "colitur: cannot read schema %s\n" p; + exit 2 + | Ok s -> s) + in + let written = ref [] in + let emit rel contents = + write_file (Filename.concat out rel) contents; + written := rel :: !written + in + for y = from_y to to_y do + let days = resolved_year_days ~overlays y in + let v = Colitur_render.View.of_days ~vocab:Rite_ef.Vocab_ef.vocab ~rite:"ef" ~year:y days in + let ys = string_of_int y in + emit ("ef/" ^ ys ^ ".json") (Colitur_render.Emit_json.year v); + emit ("ef/" ^ ys ^ ".csv") (Colitur_render.Emit_csv.year v); + emit ("ef/" ^ ys ^ ".xml") (Colitur_render.Emit_xml.year v); + emit ("ef/" ^ ys ^ ".ics") (Colitur_render.Emit_ics.year ?dtstamp v); + (* One file per day: the static equivalent of a per-day endpoint. + [View.of_days] with a one-day list yields 12 months, 11 empty, one + populated -- exactly the shape a single day's own page needs. *) + List.iter + (fun d -> + let iso = D.to_iso8601 d.Colitur_kernel.Liturgical_day.date in + let mm = String.sub iso 5 2 and dd = String.sub iso 8 2 in + let one = Colitur_render.View.of_days ~vocab:Rite_ef.Vocab_ef.vocab ~rite:"ef" ~year:y [ d ] in + emit (Printf.sprintf "ef/%s/%s/%s.json" ys mm dd) (Colitur_render.Emit_json.year one)) + days + done; + emit "schema/day-v1.json" schema; + emit "index.html" (index_html ~from_y ~to_y); + let now = List.sort compare !written in + if prune then + List.iter + (fun old -> + if not (List.mem old now) then begin + let p = Filename.concat out old in + if Sys.file_exists p then begin + Sys.remove p; + prune_empty_dirs ~out (Filename.dirname p) + end + end) + (read_manifest out); + write_file (Filename.concat out manifest_name) (String.concat "\n" now ^ "\n"); + Printf.printf "colitur: wrote %d files to %s\n" (List.length now) out + (* Help and usage are deliberately DIFFERENT things, and the difference is the Unix convention rather than a preference: asking for help is a request that SUCCEEDED, so [--help] prints to stdout and exits 0 (it can be piped into a @@ -550,6 +727,11 @@ usage: compute year Y and render it through FILE, a logic-less Mustache-family template; table and render are the same operation, two names (see "rendering" below) + colitur publish --from Y --to Y --out DIR [--overlay FILE ...] [--prune] + [--dtstamp S] + write the static tree: per-year csv/json/xml/ics, one JSON + file per day, the schema and a generated index (see + "publish" below) colitur new-overlay print a starter overlay file to stdout colitur convert FILE.ini flat INI overlay -> S-expression, on stdout colitur check FILE ... load an overlay, say what it does, exit 2 if not @@ -661,6 +843,32 @@ rendering: that JSON is the OUTPUT, never something colitur itself parses back in. +publish: + --out DIR (required) writes the static tree that IS this program's API: + any web server or git repo can serve it as-is, and nothing runs + at request time. + + ef/.{json,csv,xml,ics} one civil year, all days + ef///
.json one file per day + schema/day-v1.json the published JSON contract + index.html a generated index page + .colitur-manifest every path this run wrote + + Deterministic: publishing the same --from/--to range twice + produces a byte-identical tree (--dtstamp behaves exactly as on + `emit`). That is what makes publishing into a git repo safe -- + `git status` shows only real change, and you review an actual + diff before pushing. + + Non-destructive: publish writes only files it owns, and records + every one in .colitur-manifest. A file you put in the output + directory yourself is never in that manifest, so it is never + touched, whether or not --prune is given. --prune additionally + removes manifest entries from a PREVIOUS run that this run did + not rewrite (e.g. an earlier year's per-day files, when you + publish a different range into the same directory) -- never + anything the manifest does not name. + environment: COLITUR_DATA_DIR Read the calendar data from this directory instead of the @@ -712,6 +920,10 @@ let with_year ys f = valued, the same shape as [format]/[from_y]/[to_y]/[dtstamp] above -- `table`/`render` take one year and one template file, never a range or a repeatable list. *) +(* [out] (Task 12, `publish`) is single-valued like [format]/[year]/etc. + [prune] is the one plain boolean flag in this whole record -- every other + field here takes a value, but [--prune] does not, so it cannot reuse the + `"--flag" :: v :: rest` shape the value-taking flags share below. *) type parsed_args = { overlays : string list; format : string option; @@ -721,6 +933,8 @@ type parsed_args = { year : string option; template : string option; flavour : string option; + out : string option; + prune : bool; positional : string list; } @@ -743,6 +957,9 @@ let parse_args argv = | [ "--template" ] -> Error "--template needs a value" | "--flavour" :: v :: rest -> go { acc with flavour = Some v } rest | [ "--flavour" ] -> Error "--flavour needs a value" + | "--out" :: v :: rest -> go { acc with out = Some v } rest + | [ "--out" ] -> Error "--out needs a directory path" + | "--prune" :: rest -> go { acc with prune = true } rest (* The recognised bare flags pass through as positional words for the dispatch below to match; anything else beginning with '-' is rejected rather than silently treated as a command or a year. *) @@ -755,7 +972,7 @@ let parse_args argv = in go { overlays = []; format = None; from_y = None; to_y = None; dtstamp = None; year = None; - template = None; flavour = None; positional = [] } + template = None; flavour = None; out = None; prune = false; positional = [] } argv (* Sibling to [reject_overlays_for]: `emit`'s own four flags have no meaning @@ -792,6 +1009,27 @@ let reject_table_flags_for cmd ~year ~template ~flavour = exit 2 end +(* Sibling to [reject_emit_flags_for]/[reject_table_flags_for]: `--format` + has no meaning on `publish` (it always writes all four whole-year + formats plus the per-day JSON tree, never a single chosen one), so + accepting and silently dropping it would be the same failure mode this + project already refuses everywhere else. Narrower than + [reject_emit_flags_for] on purpose -- `publish` legitimately takes + --from/--to/--dtstamp, so that blanket check cannot be reused here. *) +let reject_format_for cmd format = + if format <> None then begin + Printf.eprintf "colitur: --format has no effect on `%s`; refusing rather than ignoring it\n" cmd; + exit 2 + end + +(* Sibling to the three rejectors above: `--out`/`--prune` (Task 12) have no + meaning on any command except `publish`. *) +let reject_publish_flags_for cmd ~out ~prune = + if out <> None || prune then begin + Printf.eprintf "colitur: --out/--prune have no effect on `%s`; refusing rather than ignoring them\n" cmd; + exit 2 + end + (* `colitur check FILE...` -- load a user overlay, apply it to the real shipped calendar, and say what it did, without printing a year of output. @@ -956,57 +1194,68 @@ let () = | Error msg -> Printf.eprintf "colitur: %s\n" msg; usage () - | Ok { overlays; format; from_y; to_y; dtstamp; year; template; flavour; positional } -> ( + | Ok { overlays; format; from_y; to_y; dtstamp; year; template; flavour; out; prune; positional } -> ( let reject_emit = reject_emit_flags_for ~format ~from_y ~to_y ~dtstamp in let reject_table = reject_table_flags_for ~year ~template ~flavour in + let reject_publish = reject_publish_flags_for ~out ~prune in match positional with | [ ("-h" | "--help" | "help") ] -> reject_overlays_for "--help" overlays; reject_emit "--help"; reject_table "--help"; + reject_publish "--help"; print_help () | [ ("-V" | "--version" | "version") ] -> reject_overlays_for "--version" overlays; reject_emit "--version"; reject_table "--version"; + reject_publish "--version"; print_endline version; exit 0 | [ "easter"; ys ] -> reject_overlays_for "easter" overlays; reject_emit "easter"; reject_table "easter"; + reject_publish "easter"; with_year ys easter_report | [ "temporal"; ys ] -> reject_overlays_for "temporal" overlays; reject_emit "temporal"; reject_table "temporal"; + reject_publish "temporal"; with_year ys temporal_report | "check" :: (_ :: _ as files) -> reject_overlays_for "check" overlays; reject_emit "check"; reject_table "check"; + reject_publish "check"; check_report files | [ "convert"; path ] -> reject_overlays_for "convert" overlays; reject_emit "convert"; reject_table "convert"; + reject_publish "convert"; convert_report path | [ "new-overlay" ] -> reject_overlays_for "new-overlay" overlays; reject_emit "new-overlay"; reject_table "new-overlay"; + reject_publish "new-overlay"; print_string new_overlay_template; exit 0 | [ "day"; ys ] -> reject_emit "day"; reject_table "day"; + reject_publish "day"; with_year ys (day_report ~overlays) | [ "readings"; ys ] -> reject_emit "readings"; reject_table "readings"; + reject_publish "readings"; with_year ys (readings_report ~overlays) | [ "emit" ] -> ( reject_table "emit"; + reject_publish "emit"; match format with | None -> Printf.eprintf "colitur: emit requires --format csv|json|sexp|xml|ics\n"; @@ -1021,6 +1270,7 @@ let () = with_year to_ys (fun to_y -> emit_report ~format ~overlays ~dtstamp ~from_y ~to_y)))) | [ ("table" | "render") as cmd ] -> ( reject_emit cmd; + reject_publish cmd; match template with | None -> Printf.eprintf "colitur: %s requires --year YEAR and --template FILE\n" cmd; @@ -1032,4 +1282,19 @@ let () = exit 2 | Some ys -> with_year ys (fun y -> table_report ~template ~flavour_opt:flavour ~overlays y) )) + | [ "publish" ] -> ( + reject_table "publish"; + reject_format_for "publish" format; + match out with + | None -> + Printf.eprintf "colitur: publish requires --out DIR\n"; + exit 2 + | Some out -> ( + match (from_y, to_y) with + | None, _ | _, None -> + Printf.eprintf "colitur: publish requires --from YEAR and --to YEAR\n"; + exit 2 + | Some from_ys, Some to_ys -> + with_year from_ys (fun from_y -> + with_year to_ys (fun to_y -> publish_report ~from_y ~to_y ~out ~overlays ~dtstamp ~prune)))) | _ -> usage ()) diff --git a/dune b/dune index b7cd19b..e2353df 100644 --- a/dune +++ b/dune @@ -23,10 +23,22 @@ ; this file was originally written to close applied to it too -- a fresh ; `dune build` left it absent from _build/default/data/ef/, only ever ; materialised there as a side effect of test/dune's own deps. +; +; schema/day-v1.json added here for the identical reason again (Task 12, +; `colitur publish`): unlike data/ef/, the top-level schema/ directory has +; no dune file of its own, so nothing makes dune copy it into +; _build/default/ by default -- confirmed empirically, a clean `dune build` +; left _build/default/schema/ missing entirely. bin/main.ml's own +; [schema_path] resolves it the same prefix-relative way [data_dir] resolves +; the sanctoral data, and that resolution needs the file actually present in +; the build tree, not only in the source tree. This is a BUILD-TIME +; convenience only -- it says nothing about `dune install`, which is +; Task 13's own job (see schema_path's comment in bin/main.ml). (alias (name default) (deps (alias_rec install) data/ef/sanctoral.sexp data/ef/adjustments.sexp - data/ef/lectionary.sexp)) + data/ef/lectionary.sexp + schema/day-v1.json)) diff --git a/man/colitur.1 b/man/colitur.1 index da08a36..d015a49 100644 --- a/man/colitur.1 +++ b/man/colitur.1 @@ -28,6 +28,15 @@ colitur \- deterministic liturgical calendar and lectionary engine (Roman rite, .RB [ \-\-overlay " FILE" " ...]" .br .B colitur +.B publish +.BI \-\-from " YEAR" +.BI \-\-to " YEAR" +.BI \-\-out " DIR" +.RB [ \-\-overlay " FILE" " ...]" +.RB [ \-\-prune ] +.RB [ \-\-dtstamp " STAMP" ] +.br +.B colitur .BR \-h | \-\-help .SH DESCRIPTION .B colitur @@ -98,6 +107,14 @@ are the same operation under two names \(em see below for why there is no separate, stdin\-fed .B render . .TP +.B publish +Write the static tree that +.I is +this program's API: a civil\-year range rendered once, as files, so any web +server or git repository can serve it and nothing runs at request time. See +.B PUBLISH +below. +.TP .BI convert " FILE" .ini Convert a flat INI overlay to the S\-expression form, on standard output. The conversion verifies its own output before emitting it: the generated text is @@ -130,7 +147,7 @@ below. .TP .BI \-\-overlay " FILE" Apply a user calendar on top of the shipped one. Repeatable and ordered; -.BR day ", " readings ", " emit ", " table " and " render +.BR day ", " readings ", " emit ", " table ", " render " and " publish only. See .B OVERLAYS below. @@ -144,7 +161,7 @@ Required. See below. .TP .BI \-\-from " YEAR" ", " \-\-to " YEAR" -.RB ( "colitur emit" " only)" +.RB ( "colitur emit" " and " "colitur publish" " only)" The inclusive civil\-year range to render, each .B 1583..9999 as elsewhere. @@ -154,7 +171,7 @@ must not be after Both required. .TP .BI \-\-dtstamp " STAMP" -.RB ( "colitur emit \-\-format ics" " only)" +.RB ( "colitur emit \-\-format ics" " and " "colitur publish" " only)" Fix the feed's own DTSTAMP instead of the default .IR YYYY0101T000000Z , where @@ -163,6 +180,26 @@ is the emitted year. Never a clock read either way \(em see .B EMIT below. .TP +.BI \-\-out " DIR" +.RB ( "colitur publish" " only)" +The directory to write the static tree into. Created if it does not exist. +Required. See +.B PUBLISH +below. +.TP +.B \-\-prune +.RB ( "colitur publish" " only)" +Remove files a previous +.B publish +run into the same +.B \-\-out +wrote that this run did not rewrite. Never removes a file that is not +recorded in +.IR out /.colitur\-manifest , +regardless of this flag. See +.B PUBLISH +below. +.TP .BI \-\-year " YEAR" .RB ( "colitur table" " and " "colitur render" " only)" The civil year to compute, @@ -472,6 +509,109 @@ crash \(em a template is user input, exactly like an overlay file. colitur: template bad.txt: unclosed section {{#days}} .fi .RE +.SH PUBLISH +.BI "colitur publish " \-\-from " YEAR " \-\-to " YEAR " \-\-out " DIR" +writes the static tree that +.I is +this program's API: every file a civil\-year range can be asked for, +computed once and written out, so any web server or git repository can +serve the result as\-is and nothing runs at request time. +.RS +.nf + +ef/.json one civil year, all days, whole\-year emitters +ef/.csv +ef/.xml +ef/.ics +ef///
.json one file per day +schema/day\-v1.json the published JSON contract +index.html a generated index page, not a template +\&.colitur\-manifest every path this run wrote, one per line +.fi +.RE +.PP +Every emitted file goes through the same emitters +.B emit +uses; a published +.I .ics +file for a given year is byte\-for\-byte what +.B "colitur emit \-\-format ics" +would print for that year, and +.B \-\-dtstamp +means exactly what it means there. The per\-day JSON files carry the same +shape as the whole\-year one, scoped to a single day \(em +.B "colitur table" +and template authors needing one day's data can read either. +.PP +.B Deterministic. +Publishing the same +.B \-\-from / \-\-to +range into an empty directory twice produces a byte\-identical tree. Nothing +in the publish path reads the wall clock; the +.I .ics +files' own DTSTAMP defaults to a fixed value derived from the emitted year, +exactly as it does under +.B emit +(see +.B EMIT +above), and +.B \-\-dtstamp +overrides it the same way. This is what makes publishing into a git +repository safe: +.B git status +shows only genuine change, and you review an actual diff before pushing, +never a rewrite of files that did not change. +.PP +.B Non\-destructive. +.B publish +writes only files it owns, and records the relative path of every one of +them in +.IR out /.colitur\-manifest +(itself never subject to pruning). A file you put in the output directory +yourself \(em by hand, or from some other tool \(em is never named in that +manifest, so it is never touched, +.I whether or not +.B \-\-prune +is given. +.RS +.nf + +.B "touch out/MY\-NOTES.txt" +.B "colitur publish \-\-from 2027 \-\-to 2027 \-\-out out \-\-prune" +.B "test \-f out/MY\-NOTES.txt && echo kept" +kept +.fi +.RE +.PP +.B \-\-prune +removes exactly the entries a +.I previous +publish into the same +.B \-\-out +wrote that this run did not rewrite \(em typically an earlier year's own +per\-day files, when a later +.B publish +targets a different +.B \-\-from / \-\-to +range into the same directory. A directory a stale entry's removal leaves +empty is removed too (so, for example, +.I out/ef/2027/ +itself goes away once every file under it is gone), but nothing above +.B \-\-out +is ever touched, and +.B \-\-out +itself is never removed even when nothing is left in it. Without +.BR \-\-prune , +old entries are left in place, and only the manifest is rewritten to +describe the current run. +.PP +.BR \-\-overlay +is accepted exactly as on +.BR day ", " readings " and " emit : +applied on top of the shipped calendar, in order, before each year in the +range is rendered. See +.B OVERLAYS +below. .SH OVERLAYS .TP .BI \-\-overlay " FILE" @@ -665,6 +805,17 @@ Render a year through a template, flavour inferred from the extension: .B colitur table \-\-year 2026 \-\-template booklet.tex > booklet.tex.out .fi .RE +.PP +Publish a year range as a static tree, then keep it in step with +.B \-\-prune +as the range moves: +.RS +.nf + +.B colitur publish \-\-from 2026 \-\-to 2027 \-\-out ~/public/colitur +.B colitur publish \-\-from 2027 \-\-to 2028 \-\-out ~/public/colitur \-\-prune +.fi +.RE .SH SOURCES The calendar is computed against the 1962 .I Missale Romanum diff --git a/test/cli.t b/test/cli.t index c5aeb98..c74331b 100644 --- a/test/cli.t +++ b/test/cli.t @@ -562,3 +562,445 @@ rather than silently ignored: $ colitur emit --format csv --from 2027 --to 2027 --year 2028 colitur: --year/--template/--flavour have no effect on `emit`; refusing rather than ignoring them [2] + +publish writes the documented tree (Task 12): the manifest itself +(.colitur-manifest) is a real file `find` sees too, since it lives in the +same directory as everything else it tracks. Every /tmp/pub* path below is +cleared first, so this section is self-contained across repeat runs: + + $ rm -rf /tmp/pub /tmp/pub1 /tmp/pub2 /tmp/pub3 + + $ colitur publish --from 2027 --to 2027 --out /tmp/pub >/dev/null + $ find /tmp/pub -type f | sed 's|/tmp/pub/||' | sort + .colitur-manifest + ef/2027.csv + ef/2027.ics + ef/2027.json + ef/2027.xml + ef/2027/01/01.json + ef/2027/01/02.json + ef/2027/01/03.json + ef/2027/01/04.json + ef/2027/01/05.json + ef/2027/01/06.json + ef/2027/01/07.json + ef/2027/01/08.json + ef/2027/01/09.json + ef/2027/01/10.json + ef/2027/01/11.json + ef/2027/01/12.json + ef/2027/01/13.json + ef/2027/01/14.json + ef/2027/01/15.json + ef/2027/01/16.json + ef/2027/01/17.json + ef/2027/01/18.json + ef/2027/01/19.json + ef/2027/01/20.json + ef/2027/01/21.json + ef/2027/01/22.json + ef/2027/01/23.json + ef/2027/01/24.json + ef/2027/01/25.json + ef/2027/01/26.json + ef/2027/01/27.json + ef/2027/01/28.json + ef/2027/01/29.json + ef/2027/01/30.json + ef/2027/01/31.json + ef/2027/02/01.json + ef/2027/02/02.json + ef/2027/02/03.json + ef/2027/02/04.json + ef/2027/02/05.json + ef/2027/02/06.json + ef/2027/02/07.json + ef/2027/02/08.json + ef/2027/02/09.json + ef/2027/02/10.json + ef/2027/02/11.json + ef/2027/02/12.json + ef/2027/02/13.json + ef/2027/02/14.json + ef/2027/02/15.json + ef/2027/02/16.json + ef/2027/02/17.json + ef/2027/02/18.json + ef/2027/02/19.json + ef/2027/02/20.json + ef/2027/02/21.json + ef/2027/02/22.json + ef/2027/02/23.json + ef/2027/02/24.json + ef/2027/02/25.json + ef/2027/02/26.json + ef/2027/02/27.json + ef/2027/02/28.json + ef/2027/03/01.json + ef/2027/03/02.json + ef/2027/03/03.json + ef/2027/03/04.json + ef/2027/03/05.json + ef/2027/03/06.json + ef/2027/03/07.json + ef/2027/03/08.json + ef/2027/03/09.json + ef/2027/03/10.json + ef/2027/03/11.json + ef/2027/03/12.json + ef/2027/03/13.json + ef/2027/03/14.json + ef/2027/03/15.json + ef/2027/03/16.json + ef/2027/03/17.json + ef/2027/03/18.json + ef/2027/03/19.json + ef/2027/03/20.json + ef/2027/03/21.json + ef/2027/03/22.json + ef/2027/03/23.json + ef/2027/03/24.json + ef/2027/03/25.json + ef/2027/03/26.json + ef/2027/03/27.json + ef/2027/03/28.json + ef/2027/03/29.json + ef/2027/03/30.json + ef/2027/03/31.json + ef/2027/04/01.json + ef/2027/04/02.json + ef/2027/04/03.json + ef/2027/04/04.json + ef/2027/04/05.json + ef/2027/04/06.json + ef/2027/04/07.json + ef/2027/04/08.json + ef/2027/04/09.json + ef/2027/04/10.json + ef/2027/04/11.json + ef/2027/04/12.json + ef/2027/04/13.json + ef/2027/04/14.json + ef/2027/04/15.json + ef/2027/04/16.json + ef/2027/04/17.json + ef/2027/04/18.json + ef/2027/04/19.json + ef/2027/04/20.json + ef/2027/04/21.json + ef/2027/04/22.json + ef/2027/04/23.json + ef/2027/04/24.json + ef/2027/04/25.json + ef/2027/04/26.json + ef/2027/04/27.json + ef/2027/04/28.json + ef/2027/04/29.json + ef/2027/04/30.json + ef/2027/05/01.json + ef/2027/05/02.json + ef/2027/05/03.json + ef/2027/05/04.json + ef/2027/05/05.json + ef/2027/05/06.json + ef/2027/05/07.json + ef/2027/05/08.json + ef/2027/05/09.json + ef/2027/05/10.json + ef/2027/05/11.json + ef/2027/05/12.json + ef/2027/05/13.json + ef/2027/05/14.json + ef/2027/05/15.json + ef/2027/05/16.json + ef/2027/05/17.json + ef/2027/05/18.json + ef/2027/05/19.json + ef/2027/05/20.json + ef/2027/05/21.json + ef/2027/05/22.json + ef/2027/05/23.json + ef/2027/05/24.json + ef/2027/05/25.json + ef/2027/05/26.json + ef/2027/05/27.json + ef/2027/05/28.json + ef/2027/05/29.json + ef/2027/05/30.json + ef/2027/05/31.json + ef/2027/06/01.json + ef/2027/06/02.json + ef/2027/06/03.json + ef/2027/06/04.json + ef/2027/06/05.json + ef/2027/06/06.json + ef/2027/06/07.json + ef/2027/06/08.json + ef/2027/06/09.json + ef/2027/06/10.json + ef/2027/06/11.json + ef/2027/06/12.json + ef/2027/06/13.json + ef/2027/06/14.json + ef/2027/06/15.json + ef/2027/06/16.json + ef/2027/06/17.json + ef/2027/06/18.json + ef/2027/06/19.json + ef/2027/06/20.json + ef/2027/06/21.json + ef/2027/06/22.json + ef/2027/06/23.json + ef/2027/06/24.json + ef/2027/06/25.json + ef/2027/06/26.json + ef/2027/06/27.json + ef/2027/06/28.json + ef/2027/06/29.json + ef/2027/06/30.json + ef/2027/07/01.json + ef/2027/07/02.json + ef/2027/07/03.json + ef/2027/07/04.json + ef/2027/07/05.json + ef/2027/07/06.json + ef/2027/07/07.json + ef/2027/07/08.json + ef/2027/07/09.json + ef/2027/07/10.json + ef/2027/07/11.json + ef/2027/07/12.json + ef/2027/07/13.json + ef/2027/07/14.json + ef/2027/07/15.json + ef/2027/07/16.json + ef/2027/07/17.json + ef/2027/07/18.json + ef/2027/07/19.json + ef/2027/07/20.json + ef/2027/07/21.json + ef/2027/07/22.json + ef/2027/07/23.json + ef/2027/07/24.json + ef/2027/07/25.json + ef/2027/07/26.json + ef/2027/07/27.json + ef/2027/07/28.json + ef/2027/07/29.json + ef/2027/07/30.json + ef/2027/07/31.json + ef/2027/08/01.json + ef/2027/08/02.json + ef/2027/08/03.json + ef/2027/08/04.json + ef/2027/08/05.json + ef/2027/08/06.json + ef/2027/08/07.json + ef/2027/08/08.json + ef/2027/08/09.json + ef/2027/08/10.json + ef/2027/08/11.json + ef/2027/08/12.json + ef/2027/08/13.json + ef/2027/08/14.json + ef/2027/08/15.json + ef/2027/08/16.json + ef/2027/08/17.json + ef/2027/08/18.json + ef/2027/08/19.json + ef/2027/08/20.json + ef/2027/08/21.json + ef/2027/08/22.json + ef/2027/08/23.json + ef/2027/08/24.json + ef/2027/08/25.json + ef/2027/08/26.json + ef/2027/08/27.json + ef/2027/08/28.json + ef/2027/08/29.json + ef/2027/08/30.json + ef/2027/08/31.json + ef/2027/09/01.json + ef/2027/09/02.json + ef/2027/09/03.json + ef/2027/09/04.json + ef/2027/09/05.json + ef/2027/09/06.json + ef/2027/09/07.json + ef/2027/09/08.json + ef/2027/09/09.json + ef/2027/09/10.json + ef/2027/09/11.json + ef/2027/09/12.json + ef/2027/09/13.json + ef/2027/09/14.json + ef/2027/09/15.json + ef/2027/09/16.json + ef/2027/09/17.json + ef/2027/09/18.json + ef/2027/09/19.json + ef/2027/09/20.json + ef/2027/09/21.json + ef/2027/09/22.json + ef/2027/09/23.json + ef/2027/09/24.json + ef/2027/09/25.json + ef/2027/09/26.json + ef/2027/09/27.json + ef/2027/09/28.json + ef/2027/09/29.json + ef/2027/09/30.json + ef/2027/10/01.json + ef/2027/10/02.json + ef/2027/10/03.json + ef/2027/10/04.json + ef/2027/10/05.json + ef/2027/10/06.json + ef/2027/10/07.json + ef/2027/10/08.json + ef/2027/10/09.json + ef/2027/10/10.json + ef/2027/10/11.json + ef/2027/10/12.json + ef/2027/10/13.json + ef/2027/10/14.json + ef/2027/10/15.json + ef/2027/10/16.json + ef/2027/10/17.json + ef/2027/10/18.json + ef/2027/10/19.json + ef/2027/10/20.json + ef/2027/10/21.json + ef/2027/10/22.json + ef/2027/10/23.json + ef/2027/10/24.json + ef/2027/10/25.json + ef/2027/10/26.json + ef/2027/10/27.json + ef/2027/10/28.json + ef/2027/10/29.json + ef/2027/10/30.json + ef/2027/10/31.json + ef/2027/11/01.json + ef/2027/11/02.json + ef/2027/11/03.json + ef/2027/11/04.json + ef/2027/11/05.json + ef/2027/11/06.json + ef/2027/11/07.json + ef/2027/11/08.json + ef/2027/11/09.json + ef/2027/11/10.json + ef/2027/11/11.json + ef/2027/11/12.json + ef/2027/11/13.json + ef/2027/11/14.json + ef/2027/11/15.json + ef/2027/11/16.json + ef/2027/11/17.json + ef/2027/11/18.json + ef/2027/11/19.json + ef/2027/11/20.json + ef/2027/11/21.json + ef/2027/11/22.json + ef/2027/11/23.json + ef/2027/11/24.json + ef/2027/11/25.json + ef/2027/11/26.json + ef/2027/11/27.json + ef/2027/11/28.json + ef/2027/11/29.json + ef/2027/11/30.json + ef/2027/12/01.json + ef/2027/12/02.json + ef/2027/12/03.json + ef/2027/12/04.json + ef/2027/12/05.json + ef/2027/12/06.json + ef/2027/12/07.json + ef/2027/12/08.json + ef/2027/12/09.json + ef/2027/12/10.json + ef/2027/12/11.json + ef/2027/12/12.json + ef/2027/12/13.json + ef/2027/12/14.json + ef/2027/12/15.json + ef/2027/12/16.json + ef/2027/12/17.json + ef/2027/12/18.json + ef/2027/12/19.json + ef/2027/12/20.json + ef/2027/12/21.json + ef/2027/12/22.json + ef/2027/12/23.json + ef/2027/12/24.json + ef/2027/12/25.json + ef/2027/12/26.json + ef/2027/12/27.json + ef/2027/12/28.json + ef/2027/12/29.json + ef/2027/12/30.json + ef/2027/12/31.json + index.html + schema/day-v1.json + + $ ls /tmp/pub/ef/2027/01/*.json | wc -l + 31 + + $ test -f /tmp/pub/schema/day-v1.json && echo schema-present + schema-present + + $ test -f /tmp/pub/index.html && echo index-present + index-present + +Publishing twice is byte-identical -- safe to publish into a git repo: + + $ colitur publish --from 2027 --to 2027 --out /tmp/pub1 >/dev/null + $ colitur publish --from 2027 --to 2027 --out /tmp/pub2 >/dev/null + $ diff -r /tmp/pub1 /tmp/pub2 && echo identical + identical + +The published .ics is byte-identical to `emit --format ics` for the same +year -- both walk through the identical Emit_ics.year: + + $ colitur emit --format ics --from 2027 --to 2027 > /tmp/emit-2027.ics + $ diff /tmp/pub1/ef/2027.ics /tmp/emit-2027.ics && echo ics-identical + ics-identical + +publish never deletes a file it does not own: + + $ touch /tmp/pub1/MY-NOTES.txt + $ colitur publish --from 2027 --to 2027 --out /tmp/pub1 >/dev/null + $ test -f /tmp/pub1/MY-NOTES.txt && echo kept + kept + +--prune removes only files a previous run created: + + $ colitur publish --from 2027 --to 2027 --out /tmp/pub1 --prune >/dev/null + $ test -f /tmp/pub1/MY-NOTES.txt && echo still-kept + still-kept + + $ colitur publish --from 2028 --to 2028 --out /tmp/pub1 --prune >/dev/null + $ test -d /tmp/pub1/ef/2027 || echo pruned-2027 + pruned-2027 + $ test -f /tmp/pub1/MY-NOTES.txt && echo notes-survived-prune + notes-survived-prune + +--out is required: + + $ colitur publish --from 2027 --to 2027 + colitur: publish requires --out DIR + [2] + +publish's own flags have no effect on the other commands, and other +commands' flags have no effect on publish -- refused rather than silently +ignored, the same discipline as everywhere else: + + $ colitur publish --from 2027 --to 2027 --out /tmp/pub3 --format json + colitur: --format has no effect on `publish`; refusing rather than ignoring it + [2] + + $ colitur day 2027 --out /tmp/pub3 --prune + colitur: --out/--prune have no effect on `day`; refusing rather than ignoring them + [2] diff --git a/test/dune b/test/dune index aa4bd66..8d0c551 100644 --- a/test/dune +++ b/test/dune @@ -43,4 +43,10 @@ ../data/ef/sanctoral.sexp ../data/ef/adjustments.sexp ../data/ef/lectionary.sexp - ../data/ef/commons.sexp)) + ../data/ef/commons.sexp + ; Task 12, `colitur publish`: the cram sandbox only ever gets what this + ; stanza names explicitly (unlike a plain `dune build`, it does not fall + ; back to the workspace root's own default alias), so schema/day-v1.json + ; needs its own entry here too, exactly like every other runtime file + ; above. + ../schema/day-v1.json)) -- cgit v1.3 From bca7dabd2b436b8fe8de21736c59437b5ea990af Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 19 Aug 2026 10:42:22 +0200 Subject: fix(cli): publish --prune refuses a manifest entry that escapes --out CRITICAL: .colitur-manifest lives INSIDE the tree publish writes into -- the very tree this feature exists to have committed into a git repo. A manifest entry with a ".." path component, or an absolute path, let --prune Sys.remove/Unix.rmdir a file OUTSIDE --out. No attacker is required: an ordinary bad merge, a conflict resolved the wrong way, or a hand-edit of that file is enough to plant such an entry, and publish's own stated contract -- it never deletes a file it does not own -- broke outright the moment one was present. Two independent checks, both required, applied before every deletion: - structural (manifest_entry_is_safe): reject an entry that is absolute or has a ".." path COMPONENT, by splitting on '/' and comparing components, not by substring-matching ".." (which would wrongly reject a legitimate name like foo..bar). - containment (resolves_under): resolve both --out and the candidate with Unix.realpath (closing a symlink-inside-out gap the structural check alone would miss) and verify the candidate is a genuine path descendant of --out, not merely a string with the same prefix. Applied at both the file-deletion loop and prune_empty_dirs' own directory removals. A rejected entry is skipped with a one-line stderr warning; publish completes rather than aborting -- a corrupted manifest must not make the tool itself unusable. test/cli.t reproduces the exact canary scenario (a ".." entry surviving deletion of a file outside --out), an absolute-path entry, and a legitimate dotted filename (no .. component) still pruning normally, alongside the existing --prune coverage. --- bin/main.ml | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++++++------ test/cli.t | 48 ++++++++++++++++++++++++++++++ 2 files changed, 138 insertions(+), 9 deletions(-) (limited to 'test/cli.t') diff --git a/bin/main.ml b/bin/main.ml index 4f1fbf2..6f3cbd5 100644 --- a/bin/main.ml +++ b/bin/main.ml @@ -561,6 +561,63 @@ let read_manifest out = | Error _ -> [] | Ok contents -> String.split_on_char '\n' contents |> List.filter (fun l -> l <> "") +(* Fix round 1 (coordinator review), CRITICAL: a manifest entry is + UNTRUSTED input the moment [--prune] reads it back. The manifest is a + plain-text file that lives INSIDE the very tree this feature exists to + have committed into a git repo -- an ordinary bad merge or a hand-edit is + enough to put an arbitrary path in it, no attacker required. Without a + check, an entry like "../outside/CANARY.txt" resolves, via + [Filename.concat out entry], to a path OUTSIDE [out], and the prune loop + below would [Sys.remove] it -- deleting a file [publish] never wrote, + breaking the "never deletes a file it does not own" contract outright. + + Two independent checks, deliberately, because either alone is easy to + regress later without anyone noticing in review: + + 1. STRUCTURAL ([manifest_entry_is_safe]) -- reject an entry that is + absolute, or that has a ".." path component anywhere. Split on '/' + and compare COMPONENTS, never a bare substring test: substring- + matching ".." would wrongly reject a legitimate name like + "foo..bar", which contains the two characters but has no ".." + component of its own. + 2. CONTAINMENT ([resolves_under]) -- even an entry that passes check 1 + is not trusted until the path it actually resolves to, symlinks + included, is verified to sit under [out]. [Unix.realpath] resolves + symlinks as well as "..", so this also catches an entry that a + symlink planted inside [out] could use to defeat check 1 alone. A + plain string-prefix compare is not enough by itself either: + "/tmp/pub1" is a byte-prefix of "/tmp/pub1-evil", a directory that is + not nested inside it at all, so [is_under] insists the character + right after the prefix is the path separator (or that the paths are + identical). *) +let manifest_entry_is_safe entry = + entry <> "" + && entry.[0] <> '/' + && not (List.mem ".." (String.split_on_char '/' entry)) + +let is_under ~root path = + let root = + if String.length root > 1 && root.[String.length root - 1] = '/' then + String.sub root 0 (String.length root - 1) + else root + in + String.equal path root + || (String.length path > String.length root + && String.sub path 0 (String.length root) = root + && path.[String.length root] = '/') + +(* [Unix.realpath] requires the path to exist, which is fine here: every + caller below checks [Sys.file_exists]/[Sys.readdir] first. Any failure + (missing path, dangling symlink, permission error) is treated as "not + contained" -- refuse to act rather than guess. *) +let resolves_under out p = + match Unix.realpath out with + | exception (Unix.Unix_error _ | Sys_error _) -> false + | out_real -> ( + match Unix.realpath p with + | exception (Unix.Unix_error _ | Sys_error _) -> false + | p_real -> is_under ~root:out_real p_real) + (* [--prune] deletes the FILES a stale manifest entry names, but that alone can leave their parent directories (ef///, then ef//) empty behind them -- and an empty directory still makes `test -d @@ -568,9 +625,18 @@ let read_manifest out = an old year is gone. Walk upward from each deleted file's own directory, removing it while it is empty, stopping at (never including) [out] itself: [out] is the caller's own directory, never ours to remove, even - when it is empty. *) + when it is empty. The same containment discipline as the file deletions + above applies here too ([resolves_under]), not only structurally (this + function is only ever reached via a [p] the file-deletion path already + validated, but re-checking each directory step is the belt to that + entry's braces -- see the two-layer reasoning above). *) let rec prune_empty_dirs ~out dir = - if dir <> out && String.length dir > String.length out && Sys.file_exists dir then + if + dir <> out + && String.length dir > String.length out + && Sys.file_exists dir + && resolves_under out dir + then match Sys.readdir dir with | [||] -> (try Unix.rmdir dir with Unix.Unix_error _ -> ()); @@ -679,16 +745,31 @@ let publish_report ~from_y ~to_y ~out ~overlays ~dtstamp ~prune = emit "schema/day-v1.json" schema; emit "index.html" (index_html ~from_y ~to_y); let now = List.sort compare !written in + (* Every stale entry is validated TWICE before anything is removed -- + see [manifest_entry_is_safe]/[resolves_under]'s own comment above for + why both layers exist. A rejected entry is skipped and warned about on + stderr, never fatal: a corrupt or hand-mangled manifest must not make + `publish` itself unusable -- it completes, having refused to act on + the bad line. *) if prune then List.iter (fun old -> - if not (List.mem old now) then begin - let p = Filename.concat out old in - if Sys.file_exists p then begin - Sys.remove p; - prune_empty_dirs ~out (Filename.dirname p) - end - end) + if not (List.mem old now) then + if not (manifest_entry_is_safe old) then + Printf.eprintf + "colitur: refusing to prune manifest entry %S (absolute path or .. component) +" old + else begin + let p = Filename.concat out old in + if Sys.file_exists p then + if resolves_under out p then begin + Sys.remove p; + prune_empty_dirs ~out (Filename.dirname p) + end + else + Printf.eprintf "colitur: refusing to prune %s (resolves outside %s) +" p out + end) (read_manifest out); write_file (Filename.concat out manifest_name) (String.concat "\n" now ^ "\n"); Printf.printf "colitur: wrote %d files to %s\n" (List.length now) out diff --git a/test/cli.t b/test/cli.t index c74331b..a4fa696 100644 --- a/test/cli.t +++ b/test/cli.t @@ -1004,3 +1004,51 @@ ignored, the same discipline as everywhere else: $ colitur day 2027 --out /tmp/pub3 --prune colitur: --out/--prune have no effect on `day`; refusing rather than ignoring them [2] + +--prune's manifest-driven deletion is hardened against a manifest entry it +did not itself write (fix round 1, F1, CRITICAL): the manifest lives INSIDE +the tree publish writes into, so a bad merge or a hand-edit can put an +arbitrary path in it -- no attacker required. This is the exact CANARY +reproduction the finding was raised with: a ".." entry appended to the +manifest must never let --prune delete outside --out. + + $ rm -rf /tmp/pub-sec /tmp/pub-sec-outside + $ mkdir -p /tmp/pub-sec-outside + $ touch /tmp/pub-sec-outside/CANARY.txt + $ colitur publish --from 2027 --to 2027 --out /tmp/pub-sec >/dev/null + $ echo '../pub-sec-outside/CANARY.txt' >> /tmp/pub-sec/.colitur-manifest + $ colitur publish --from 2028 --to 2028 --out /tmp/pub-sec --prune >/dev/null + colitur: refusing to prune manifest entry "../pub-sec-outside/CANARY.txt" (absolute path or .. component) + $ test -f /tmp/pub-sec-outside/CANARY.txt && echo canary-survives + canary-survives + +The same run's own legitimate stale entries (2027's files, superseded by +2028) still prune normally -- the hardening does not disable pruning, only +unsafe entries: + + $ test -d /tmp/pub-sec/ef/2027 || echo 2027-pruned-normally + 2027-pruned-normally + +An absolute-path entry is refused the same way, not only a ".." one: + + $ echo '/tmp/pub-sec-outside/CANARY.txt' >> /tmp/pub-sec/.colitur-manifest + $ colitur publish --from 2028 --to 2028 --out /tmp/pub-sec --prune >/dev/null + colitur: refusing to prune manifest entry "/tmp/pub-sec-outside/CANARY.txt" (absolute path or .. component) + $ test -f /tmp/pub-sec-outside/CANARY.txt && echo canary-still-survives + canary-still-survives + +A legitimate filename that merely CONTAINS two dots -- but has no ".." path +COMPONENT -- is not caught by the same check, proving it is not +over-broad: it still prunes normally when stale. + + $ touch /tmp/pub-sec/ef/2027..old.json + $ echo 'ef/2027..old.json' >> /tmp/pub-sec/.colitur-manifest + $ colitur publish --from 2029 --to 2029 --out /tmp/pub-sec --prune >/dev/null + $ test -f /tmp/pub-sec/ef/2027..old.json || echo dotted-name-pruned + dotted-name-pruned + +...and that same run is an ordinary --prune cycle in every other respect -- +2028's own files, now stale relative to 2029, are gone too: + + $ test -d /tmp/pub-sec/ef/2028 || echo pruned-2028 + pruned-2028 -- cgit v1.3 From b084bba86fe9d394d6a5ea287161864e7695a609 Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Wed, 19 Aug 2026 11:35:21 +0200 Subject: fix(cli): guard publish's IO, validate --dtstamp, and list all commands publish's own mkdir_p/write_file (unlike every other IO path on this branch) were unguarded: an unwritable --out parent raised a bare Unix.Unix_error(EACCES,...) and an --out naming an existing file raised ENOTDIR, both as uncaught exceptions with a stack trace rather than the project's one-line "colitur: ..." form. The same defect class commit 6bd741b already fixed once for template reads -- --out is user input too. Fixed by wrapping the whole publish_report call (not each write_file site) in one handler for Unix.Unix_error and Sys_error, mirroring why that earlier fix guarded the whole read and not only the open. Added a cram case using a read-only directory inside the test's own cram sandbox, not /tmp, so a failed cleanup cannot leave an unwritable directory behind in a shared location. --dtstamp was the only user string reaching output unescaped and unvalidated: "--dtstamp hello" silently emitted an invalid "DTSTAMP:hello", and a value carrying its own CRLF injected extra lines into every VEVENT. Fixed by rejecting anything not matching RFC 5545's UTC form (8 digits, 'T', 6 digits, 'Z') before either emit or publish does anything else, one line to stderr, exit 2. usage() was byte-unchanged from before the branch and listed only the six pre-existing commands, omitting all four commands this branch added (emit, table, render, publish). Added them; the three cram pins of the exact usage string are updated to match. --- bin/main.ml | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- test/cli.t | 43 ++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 98 insertions(+), 5 deletions(-) (limited to 'test/cli.t') diff --git a/bin/main.ml b/bin/main.ml index 6f3cbd5..6f64582 100644 --- a/bin/main.ml +++ b/bin/main.ml @@ -389,7 +389,38 @@ let readings_report ~overlays y = resolved_year_report ~line:readings_line ~over file; XML: one document per year, the schema's own root is a single year; ICS: one VCALENDAR per year, valid to concatenate for a subscriber that reads multiple files). *) +(* [--dtstamp] is the only user string that reaches [emit]/[publish] output + unescaped and unvalidated (it becomes an ICS DTSTAMP: property value + directly, Colitur_render.Emit_ics.year's own [dtstamp] parameter) -- + every OTHER interpolated value in this project's output is either + escaped (Escape.apply) or engine-computed, never raw user input placed + straight into a line-oriented format. RFC 5545 section 3.3.5 defines + DATE-TIME's UTC form as exactly 8 digits, "T", 6 digits, "Z" + (e.g. "20270101T000000Z"); rejecting anything else is what stops + "--dtstamp hello" from silently emitting a malformed "DTSTAMP:hello" AND + what stops a value carrying its own CRLF (e.g. "X\r\nBEGIN:VEVENT\r\n...") + from being injected verbatim into every VEVENT -- a value shaped exactly + like the real form cannot contain either character. *) +let dtstamp_well_formed s = + let is_digit c = c >= '0' && c <= '9' in + String.length s = 16 + && String.for_all is_digit (String.sub s 0 8) + && s.[8] = 'T' + && String.for_all is_digit (String.sub s 9 6) + && s.[15] = 'Z' + +let check_dtstamp = function + | None -> () + | Some s when dtstamp_well_formed s -> () + | Some s -> + Printf.eprintf + "colitur: --dtstamp %S is not RFC 5545 UTC form (want 8 digits, 'T', 6 digits, 'Z', e.g. \ + 20270101T000000Z)\n" + s; + exit 2 + let emit_report ~format ~overlays ~dtstamp ~from_y ~to_y = + check_dtstamp dtstamp; if from_y > to_y then begin Printf.eprintf "colitur: --from %d is after --to %d\n" from_y to_y; exit 2 @@ -695,6 +726,7 @@ let index_html ~from_y ~to_y = Buffer.contents b let publish_report ~from_y ~to_y ~out ~overlays ~dtstamp ~prune = + check_dtstamp dtstamp; if from_y > to_y then begin Printf.eprintf "colitur: --from %d is after --to %d\n" from_y to_y; exit 2 @@ -978,7 +1010,9 @@ let print_help () = let usage () = prerr_endline "colitur: usage: colitur easter | colitur temporal | colitur day | colitur \ - readings | colitur check FILE | colitur new-overlay (try: colitur --help)"; + readings | colitur emit --format FMT --from Y --to Y | colitur table --year Y --template \ + FILE | colitur render --template FILE --year Y | colitur publish --from Y --to Y --out DIR | \ + colitur check FILE | colitur new-overlay (try: colitur --help)"; exit 2 let with_year ys f = @@ -1383,5 +1417,27 @@ let () = exit 2 | Some from_ys, Some to_ys -> with_year from_ys (fun from_y -> - with_year to_ys (fun to_y -> publish_report ~from_y ~to_y ~out ~overlays ~dtstamp ~prune)))) + with_year to_ys (fun to_y -> + (* [publish_report] writes many files across a whole + year range ([mkdir_p]/[write_file], both above) -- + an unwritable [--out] parent (EACCES) or an [--out] + that names an existing plain file (ENOTDIR) raises + from deep inside that loop, same defect class as + the template read this project already guards + (commit 6bd741b): "--out" is user input too, and + the WHOLE call is guarded here rather than each + [write_file] site individually, for the same + reason that fix guarded the whole read and not + only the open. [Unix.mkdir] raises + [Unix.Unix_error] directly; [open_out_bin] + (stdlib, not the Unix module) wraps the same + underlying errno in [Sys_error] instead -- both + are real on this path, so both are caught. *) + try publish_report ~from_y ~to_y ~out ~overlays ~dtstamp ~prune with + | Unix.Unix_error (e, fn, arg) -> + Printf.eprintf "colitur: %s: %s: %s\n" fn arg (Unix.error_message e); + exit 2 + | Sys_error e -> + Printf.eprintf "colitur: %s\n" e; + exit 2)))) | _ -> usage ()) diff --git a/test/cli.t b/test/cli.t index a4fa696..80c8078 100644 --- a/test/cli.t +++ b/test/cli.t @@ -17,7 +17,7 @@ A year outside the supported domain is rejected (exit 2): No/garbage arguments give a usage error (exit 2): $ colitur - colitur: usage: colitur easter | colitur temporal | colitur day | colitur readings | colitur check FILE | colitur new-overlay (try: colitur --help) + colitur: usage: colitur easter | colitur temporal | colitur day | colitur readings | colitur emit --format FMT --from Y --to Y | colitur table --year Y --template FILE | colitur render --template FILE --year Y | colitur publish --from Y --to Y --out DIR | colitur check FILE | colitur new-overlay (try: colitur --help) [2] The EF temporal cycle for a year, one line per day: @@ -327,14 +327,14 @@ A flag needing a value, given none: $ colitur day 2026 --overlay colitur: --overlay needs a file path - colitur: usage: colitur easter | colitur temporal | colitur day | colitur readings | colitur check FILE | colitur new-overlay (try: colitur --help) + colitur: usage: colitur easter | colitur temporal | colitur day | colitur readings | colitur emit --format FMT --from Y --to Y | colitur table --year Y --template FILE | colitur render --template FILE --year Y | colitur publish --from Y --to Y --out DIR | colitur check FILE | colitur new-overlay (try: colitur --help) [2] An unknown option is rejected rather than treated as a positional word: $ colitur day 2026 --diocese colitur: unknown option --diocese - colitur: usage: colitur easter | colitur temporal | colitur day | colitur readings | colitur check FILE | colitur new-overlay (try: colitur --help) + colitur: usage: colitur easter | colitur temporal | colitur day | colitur readings | colitur emit --format FMT --from Y --to Y | colitur table --year Y --template FILE | colitur render --template FILE --year Y | colitur publish --from Y --to Y --out DIR | colitur check FILE | colitur new-overlay (try: colitur --help) [2] The shipped example overlay is runnable documentation, and it must actually @@ -464,6 +464,28 @@ An unknown format is a usage error on stderr, exit 2: colitur: unknown format "yaml" (want csv, json, sexp, xml or ics) [2] +--dtstamp is the only user string that reaches ICS output unescaped and +unvalidated -- it must be exactly RFC 5545's UTC DATE-TIME form (8 digits, +"T", 6 digits, "Z") or refused outright, rather than either silently +emitting a malformed DTSTAMP or, worse, letting an embedded CRLF inject +extra lines into every VEVENT: + + $ colitur emit --format ics --from 2027 --to 2027 --dtstamp hello + colitur: --dtstamp "hello" is not RFC 5545 UTC form (want 8 digits, 'T', 6 digits, 'Z', e.g. 20270101T000000Z) + [2] + + $ colitur emit --format ics --from 2027 --to 2027 --dtstamp "$(printf 'X\r\nBEGIN:VEVENT\r\nUID:evil')" + colitur: --dtstamp "X\r\nBEGIN:VEVENT\r\nUID:evil" is not RFC 5545 UTC form (want 8 digits, 'T', 6 digits, 'Z', e.g. 20270101T000000Z) + [2] + +A well-formed value is threaded through unchanged. (The events themselves +end in CRLF per RFC 5545 -- match the substring, not a `$`-anchored full +line, or a shell that does not mangle the trailing "\r" is doing the +grep-anchor's job for it by accident.) + + $ colitur emit --format ics --from 2027 --to 2027 --dtstamp 20270101T000000Z | grep -c 'DTSTAMP:20270101T000000Z' + 365 + emit refuses a reversed range rather than emitting nothing: $ colitur emit --format csv --from 2028 --to 2027 @@ -993,6 +1015,21 @@ publish never deletes a file it does not own: colitur: publish requires --out DIR [2] +publish writes many files across a whole year range (mkdir_p/write_file), +same as the template read guarded in commit 6bd741b -- --out is user input +too, and an unwritable parent used to surface as an uncaught +Unix.Unix_error instead of the project's one-line form. The read-only +directory below lives in this test's own cram sandbox, not /tmp: a failed +`rm -rf` of an unwritable directory would otherwise leave it behind in a +shared location, so it is restored to writable before the test ends either +way: + + $ mkdir ro-parent && chmod 555 ro-parent + $ colitur publish --from 2027 --to 2027 --out ro-parent/sub + colitur: mkdir: ro-parent/sub: Permission denied + [2] + $ chmod 755 ro-parent + publish's own flags have no effect on the other commands, and other commands' flags have no effect on publish -- refused rather than silently ignored, the same discipline as everywhere else: -- cgit v1.3