1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
|
# National Bibles Implementation Design
**Goal:** Let anyone add their nation's Bible to lectio as a formatted data file — usable either dropped into a runtime directory (no toolchain) or embedded into the shipped binary via a validated Makefile step — so offline readings can render in any language, and so a corrected Wujek can return as a pure data drop-in.
**Architecture:** A "corpus" is a pair of files sharing a basename `<code>`: `<code>.tsv` (verse text) + `<code>.ini` (metadata). The identical layout serves both built-in corpora (in `internal/bible/corpora/`, embedded via a glob `go:embed`) and user corpora (in a runtime directory, discovered at startup, overriding built-ins of the same code). One validator (`lectio --corpus-check`) gates both paths. Reading-text selection becomes config-driven (`reading_version` explicit, else language auto-match, else Latin fallback). No third-party dependencies; pure stdlib + the existing `internal/ini` reader.
**Tech Stack:** Go (stdlib + `internal/ini`), 6-column TSV corpora, INI sidecars, `go:embed`, Makefile + POSIX `sh` scripts.
## Global Constraints
- **Corpus text format:** 6 tab-separated columns, exactly, UTF-8, one row per verse: `Book · Abbrev · BookNum · Chapter · Verse · Text`. Rows whose column count != 6 are skipped by the loader (existing behaviour, unchanged).
- **`Book` column:** the canonical English book name, and it MUST be a member of the canonical book set (the 73 keys in `internal/bible/books.ini`). Lookups key on this column; `Abbrev` (col 2) and `BookNum` (col 3) are informational and ignored by the loader.
- **Metadata sidecar:** `<code>.ini`, full-line-comment INI (parsed by `internal/ini`), keys: `lang` (required), `name` (required), `psalm_system` (required: `vulgate`|`hebrew`|`drb`), `sigla` (optional, a `books.ini` dialect id).
- **Delivery:** both compile-in (glob `go:embed` + Makefile validate/embed) and runtime drop-in (a user directory, overrides built-ins). Same format and validator for both.
- **User corpus directory:** `~/.config/lectio/corpora/` (consistent with `~/.config/lectio/calendars/`; see Open Decision D1).
- **Selection order (offline reading text):** `reading_version` (explicit config) → corpus whose `lang` matches `ui_language`/`traditional_lang` → Latin `vul` fallback. Defaults preserved: en→drb, pl→vul.
- **`traditional_lang` is not extended and is deprecated:** it stays `pl`/`en`, keeps its role as the missalemeum-scraper language knob for the still-live daily view, and gains no new semantics. The corpus resolver may read it as one of the *existing* language signals (to preserve current pl/en selection behaviour), but no new field, value, or meaning is added to it, and it is removed only when the scraper is retired (separate future work). New-language selection goes through `reading_version`, never through `traditional_lang`.
- **Purity:** `internal/bible` imports only stdlib + `internal/ini` + `internal/psalter`. No network, no new third-party deps.
- **Non-breaking:** existing `--ref`, version comparison, TUI/web daily view, and the scraper path keep working unchanged.
## Detailed Design
### 1. Corpus file pair
A corpus `fr-crampon` is two files:
`fr-crampon.tsv` (verse text; `Abbrev`/`BookNum` may be placeholders):
```
Genesis Gn 1 1 1 Au commencement, Dieu créa le ciel et la terre.
Genesis Gn 1 1 2 La terre était informe et vide…
Daniel Dn 27 13 1 Il y avait à Babylone un homme du nom de Joakim.
```
`fr-crampon.ini` (metadata):
```ini
; lectio corpus metadata
lang = fr
name = Bible Crampon (français)
psalm_system = vulgate
sigla = fr
```
`psalm_system` values:
- `vulgate` — Septuagint/Vulgate psalm numbering (e.g. Ps 9 = Hebrew 9+10). Used by `vul`, `wuj`, `grb`.
- `hebrew` — Masoretic/modern numbering, no verse shift.
- `drb` — Hebrew chapter numbering with the Douay-Rheims title-fold verse shift. Used by `drb`.
Built-in corpora gain their own sidecars (`vul.ini`, `drb.ini`, `grb.ini`, `wuj.ini`) so the metadata is uniform and no version metadata is hard-coded in Go.
### 2. Discovery & loading
- **Embed:** change the directive in `internal/bible/bible.go` from the explicit file list to a glob: `//go:embed corpora/*.tsv corpora/*.ini`. Adding a pair to `corpora/` + rebuild embeds it — no directive edit.
- **Runtime:** `~/.config/lectio/corpora/` is scanned once at first corpus use. A `<code>.tsv` there defines/overrides code `<code>`; its sidecar `<code>.ini` supplies metadata (falling back to the embedded sidecar if the user omitted one for an overridden built-in).
- **Override rule:** for a given code, a runtime file wins over the embedded file (so a corrected `wuj.tsv` replaces the broken built-in). New codes simply add to the registry.
- **Registry:** a process-wide map `code → {source: embed|user, meta}` built lazily and cached under the existing `corporaMu`. `load(code)` reads the winning `.tsv` into the existing `books[book][chap][]Verse` structure (verse order preserved from file order, as today).
### 3. Corpus metadata (`internal/bible/corpusmeta.go`, new)
- `type CorpusMeta struct { Code, Lang, Name, PsalmSystem, Sigla string }`.
- `Corpora() []CorpusMeta` — all known corpora (embedded + user), sorted, user overriding embed.
- `Meta(code string) (CorpusMeta, bool)`.
- `CorporaForLang(lang string) []CorpusMeta` — for selection.
- Parsing uses `ini.Parse`; a missing sidecar for a user `.tsv` yields a `CorpusMeta` with empty fields (validator flags it; loader still serves text with `psalm_system` defaulting to `vulgate` and a warning).
### 4. Selection & fallback
- `config.Config` gains `ReadingVersion string` (`toml:"reading_version"`, INI key `reading_version`), default empty.
- New `config.Config.ReadingCorpus() string`:
1. if `ReadingVersion` set and that corpus exists → it;
2. else the first corpus (deterministic order) whose `lang` == `NormalizeUILanguage(UILanguage)`, then whose `lang` == `TraditionalLang`;
3. else `""` (caller applies the Latin `vul` fallback already implemented in `readingLine`).
- `internal/cli/liturgy.go` `vernacularVersion(cfg)` is replaced by a call to this resolver; if it returns `""`, keep the existing `drb`-for-en / `vul`-for-others defaults so current behaviour is preserved when no corpus matches.
- The existing `latinFallback = "vul"` mechanism in `readingLine` is unchanged and still covers per-verse gaps in whatever corpus is chosen.
- **Scope of lang-auto (step 2):** because `ui_language`/`traditional_lang` are `pl`/`en` only, auto-match can select a corpus only for those two languages. This is deliberate and does two useful things: it preserves today's pl/en defaults, and it lets a dropped-in corpus that declares `lang = pl` (e.g. a corrected Wujek) auto-serve Polish users with **zero config**, superseding the `vul` default. A genuinely new language (fr, de, …) has no matching signal, so it is selected only via an explicit `reading_version` — which is exactly the one line a French user adds.
### 5. Psalm versification
`internal/psalter` already maps psalm citations for `vulgate`/`drb` systems (used today via a hard-coded per-version switch). Change: the reading-render path passes the *selected corpus's* `psalm_system` (from its metadata) into the psalter mapping instead of a hard-coded table, and `hebrew` is added as the identity (no-shift) case. This fixes Vulgate-vs-Hebrew mismatches generically (the concrete `wuj` psalm-numbering problem), and lets a user corpus declare its own system.
### 6. Validation — `lectio --corpus-check <code>`
Runs against a code resolvable from either the embed or the user dir (also accepts a path for pre-embed CI use). Reports:
- **Parse:** every non-blank line has exactly 6 tab-separated fields; `Chapter`/`Verse` parse as positive ints; text is valid UTF-8 with no embedded tab/newline.
- **Book names:** every `Book` value is in the canonical set (the `books.ini` keys); lists unknown names.
- **Verse integrity, per (book, chapter):** duplicate verse numbers; non-monotonic order; intra-chapter gaps.
- **Sidecar:** `<code>.ini` present; `lang`/`name`/`psalm_system` set; `psalm_system` ∈ {vulgate,hebrew,drb}; `sigla` (if set) names an existing `books.ini` dialect.
- **Coverage report vs Latin `vul`:** missing books, missing whole chapters, and — classified by run-shape — contiguous blocks (likely real omissions) vs scattered single verses (likely different division), plus a psalm-count flag. (This is the audit already prototyped in this session, promoted to a command.)
- **Exit code:** non-zero on hard errors (bad format, unknown book, bad sidecar); zero with warnings on coverage gaps. Human-readable text output; a `--json` flag emits the same as JSON for CI.
### 7. Scripts & Makefile
- `scripts/corpus-validate.sh <code-or-path>` — thin wrapper: `go run ./cmd/lectio --corpus-check "$1"`; usable standalone (contributor / CI) before embedding. Non-zero exit propagates.
- `Makefile` targets:
- `add-corpus CORPUS=<code> SRC=<dir>` — validates the pair at `SRC/<code>.{tsv,ini}`, and only on success copies both into `internal/bible/corpora/`, then runs `build`. Validation failure aborts before any copy (so a broken Bible can never be embedded).
- `check-corpora` — runs `--corpus-check` over every embedded `<code>` (CI gate; also wired into the default `test`/`check` target).
- The runtime drop-in path needs none of these — drop the pair into `~/.config/lectio/corpora/` and run.
### 8. Surfacing user corpora
- `lectio --list` and the version-comparison surfaces (CLI/TUI/web) enumerate corpora via `bible.Corpora()` and label them with each corpus's `name`, so a user corpus appears wherever built-ins do.
- i18n `Version` labels fall back to the corpus `name` for codes not in the static i18n map.
## Testing Approach
- `internal/bible`: table tests for glob-embed loading, user-dir override precedence, metadata parsing (valid/missing/malformed sidecar), and `ReadingCorpus` resolution order.
- `internal/config`: `reading_version` round-trips through INI read/render; resolver defaults.
- `internal/cli`: `--corpus-check` on fixtures — a clean corpus, one with an unknown book, one with a verse gap, one missing a sidecar — asserting messages and exit codes; `--json` shape.
- `internal/psalter`: `hebrew` identity mapping; per-corpus system selection.
- Makefile: `check-corpora` passes for all shipped corpora (guards regressions in the built-in sidecars).
- Fixtures live under `internal/bible/testdata/corpora/`, tiny (a few books), not full bibles.
## Out of Scope (YAGNI)
- Shipping any actual national Bible (each is a later per-contribution data drop-in, including a corrected Wujek).
- Retiring the missalemeum scraper and removing `traditional_lang` (separate "migrate daily view off scraper" work; gated on deciding how much of the full EF propers to compute offline).
- Computing non-reading EF propers (Introit/Collect/Gradual/Tract/Offertory/Secret/Communion/Postcommunion).
- Auto-download of bibles, any GUI for corpus management, per-form (OF vs EF) corpus selection (the vernacular text is form-independent).
## Open Decisions
- **D1 — user corpus directory:** chosen `~/.config/lectio/corpora/` for consistency with `~/.config/lectio/calendars/` and single-place discoverability. Alternative: `~/.local/share/lectio/corpora/` (XDG-correct for multi-MB data). Revisit if the config-dir size bothers you.
- **D2 — `sigla` linkage:** a corpus may name a `books.ini` dialect for citation *display*; citation *resolution* stays English-authored (unchanged). If a corpus's language has no `books.ini` dialect, display falls back to the configured `sigla_style`. No new dialect is required to add a corpus.
|
