From 42b02c47be9b285099203e44a2570636d4ca6f03 Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Fri, 11 Sep 2026 14:47:10 +0200 Subject: krino: foundation — sexp reader, config language, init/new/check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/design.md | 626 ++++++++++++++++++++++++++++++++++++++++++++++++++++ docs/sexp-primer.md | 300 +++++++++++++++++++++++++ 2 files changed, 926 insertions(+) create mode 100644 docs/design.md create mode 100644 docs/sexp-primer.md (limited to 'docs') diff --git a/docs/design.md b/docs/design.md new file mode 100644 index 0000000..ed77e8c --- /dev/null +++ b/docs/design.md @@ -0,0 +1,626 @@ +# krino design + +Status: draft for review, 2026-09-11. Target release: **0.0.1**. + +krino (from Greek κρίνω, "to separate, to judge, to decide") sorts files in +chosen directories by rules. A rule tests a file's type, name, path, size, +age, text content or duplicate status, combined with `and` / `or` / `not`, +and applies actions: copy, move, rename, delete. krino shows what it will do, +lets you approve all of it or file by file, logs every step, and can undo a +run. + +It grew out of an earlier Python prototype; the lessons that shaped it are +in §16. + +--- + +## 1. Scope + +**0.0.1 is:** the engine and a command-line interface. + +- per-directory rule files, a main file listing which directories run, a + template for new ones +- conditions with `and` / `or` / `not` over type, name, path, content, size, + age, duplicates +- actions copy, move, rename, delete (to Trash by default), chained +- plan, review, approve all or per file, `-y` to skip review +- a log of every step, and `krino undo` +- placeholders in destinations and new names +- builds for Linux, FreeBSD and OpenBSD + +**Not in 0.0.1:** the GUI (a separate project after 0.0.1, see §12), watch +mode, OCR, EXIF dates, macOS and Windows, a content cache, a shell-command +action, looking inside archives. + +## 2. Terms + +| Term | Meaning | +|---|---| +| directory | a configured directory krino sorts, e.g. `~/downloads`; one config file each | +| root | that directory's path | +| rule | a named condition plus actions | +| test | one leaf of a condition, e.g. `(type pdf)` | +| chain | the ordered actions krino will perform on one file | +| plan | every chain for one directory, computed before anything changes | +| run | one invocation of krino that applied at least one step, across all the directories it processed; has an id; the unit of undo | + +## 3. Files and locations + +``` +$XDG_CONFIG_HOME/krino/ default ~/.config/krino + krino.conf main file: which directories run, defaults, log path + template.conf copied by `krino new` + dirs/.conf one per directory +$XDG_STATE_HOME/krino/ default ~/.local/state/krino + krino.log the log (§9) + .lock per-directory lock while a run is active +``` + +`krino init` creates the config directory with a commented `krino.conf` and +`template.conf` (both embedded in the binary). It refuses if krino.conf +exists and keeps an existing template.conf. +`krino new NAME PATH` copies `template.conf` to `dirs/NAME.conf`, fills in +the path, and appends NAME to `include` in `krino.conf`, keeping its comments. + +Generated files begin with `;; -*- mode: lisp -*-` and `;; vim: set ft=lisp :` +so editors apply Lisp highlighting and parenthesis matching. + +## 4. Config language + +### 4.1 Syntax + +The files are s-expressions (see `docs/sexp-primer.md` for a tutorial). + +- **List:** `(` items separated by whitespace `)` +- **String:** `"..."`. A backslash escapes only `"` and `\`; any other + backslash is kept literally, so `"\bacme\b"` is the regex `\bacme\b`. + Strings may span lines. +- **Symbol:** any other run of characters except whitespace, `(`, `)`, `"`, `;`. +- **Comment:** `;` to end of line. +- Encoding UTF-8. No other syntax: no quote characters, no dotted pairs, + no block comments. + +The reader records the byte offset, line and column of every list and atom. +krino edits a file by splicing text at those offsets (`krino new` inserts a +name into `include`), so everything else, comments and layout included, +stays byte for byte. A printer for generated forms arrives with the GUI. +Errors report `file:line:col`. An unclosed list is reported where it opened, +described by its first atoms: `"(" never closed: (rule "acme" ...)`. + +Paths, keywords, regexes and rule names must be strings. Settings values, +type names, operators, sizes and durations are symbols. + +### 4.2 `krino.conf` + +```lisp +(include "downloads" "documents") ; dirs/.conf, run in this order +(log "~/.local/state/krino/krino.log") ; optional +(defaults ; optional; any setting from 4.4 + (min-age 5m)) +``` + +### 4.3 `dirs/.conf` + +```lisp +(path "~/downloads") ; required +(recursive no) ; any setting from 4.4 +(ignore "*.part" "*.aria2" ".*") ; may repeat; patterns accumulate in order + +(rule "acme" + (when (type document) + (or (content "acme ltd" "0000000000") + (name "\bacme\b")) + (not (name "^draft"))) + (move "Work/Acme/{mtime:%Y}") + (stop)) +``` + +Top-level forms may appear in any order, except that rules are evaluated in +the order written. + +### 4.4 Settings + +Each setting can appear in `(defaults ...)`, at the top of a directory file, +or (where marked) inside a rule. The most specific wins: rule, then +directory, then defaults, then built-in. + +| Setting | Values | Built-in | In a rule | +|---|---|---|---| +| `case` | `ignore` \| `strict` | `ignore` | yes | +| `fold` | `yes` \| `no`. Strip diacritics before comparing: ą→a, ł→l, é→e, ü→u, and so on for all Latin letters | `yes` | yes | +| `recursive` | `yes` \| `no` | `no` | | +| `max-depth` | integer, 1 = root only | unlimited | | +| `min-age` | duration. Skip files modified more recently | `2m` | | +| `max-read` | size. No content extraction above this file size | `50M` | | +| `busy` | suffixes. Skip `f` when `f` exists beside it | `".part" ".aria2" ".crdownload"` | | +| `on-conflict` | `suffix` \| `skip` \| `overwrite` (§7.4) | `suffix` | yes | + +Durations: integer plus `s m h d w`. Sizes: integer plus optional `K M G T` +(powers of 1024). + +### 4.5 Rules + +```lisp +(rule NAME ITEM...) +``` + +Items, in any order except that actions run in the order written: + +| Item | Meaning | +|---|---| +| `(when COND...)` | the condition. Several conditions means all must hold. A rule without `when` matches every file. An empty `(when)` is an error. | +| `(case ...)` `(fold ...)` `(on-conflict ...)` | rule-level settings | +| `(copy DEST)` | copy the file into directory DEST | +| `(move DEST)` | move the file into directory DEST | +| `(rename NAME)` | rename in place; NAME must not contain `/` | +| `(delete)` | move to Trash | +| `(delete permanent)` | unlink; cannot be undone | +| `(stop)` | once this rule matches, evaluate no further rules for this file | + +A rule with only `(stop)` is an exclusion: files it matches receive no +actions from later rules. + +`DEST` is a directory: relative paths are relative to the root, `~` expands, +absolute paths are allowed. It is created if missing. `DEST` and `NAME` take +placeholders (§7.3). + +## 5. Conditions + +### 5.1 Operators + +`(and C...)`, `(or C...)`, `(not C)`. `and` and `or` take one or more +arguments; `not` exactly one. + +### 5.2 Tests + +| Test | True when | +|---|---| +| `(type T...)` | the name ends in `.T` for any T, case-insensitively. T may be a group (§A) or a multi-part suffix like `tar.gz`. | +| `(name "RE"...)` | the file name matches any regex | +| `(path "RE"...)` | the path relative to the root matches any regex | +| `(content "KW"...)` | the extracted text contains any keyword (§6) | +| `(size OP SIZE)` | OP is `> >= < <= =` | +| `(age OP DURATION)` | age by modification time | +| `(duplicate)` | another scanned file has identical content, and this one is not the original (§5.5) | +| `(duplicate "DIR"...)` | the same, also comparing against files under those directories | +| `(matched)` | an earlier rule matched this file | + +Regexes use Go's RE2 syntax: POSIX ERE plus `\d \w \s \b`, non-greedy +quantifiers, and inline flags `(?i)` `(?-i)`; no backreferences or lookaround. + +### 5.3 Case and diacritics + +`case` and `fold` apply to `name`, `path` and `content`, in both the file's +data and the pattern. `type` is always case-insensitive. A regex can +override `case` locally with `(?i)` or `(?-i)`. Folding decomposes letters +(Unicode NFD), drops combining marks, and maps the letters that do not +decompose (ł ø đ ħ ß æ œ and their capitals) to ASCII. + +### 5.4 Evaluation + +1. Every test sees **the original file**: name, path, size, mtime and + content as they were when scanning. Actions from earlier rules do not + change what later rules see. +2. Rules are evaluated in order. A matching rule adds its actions to the + file's chain. `(stop)` on a matching rule ends evaluation for that file. +3. Tests have no side effects, so `and` / `or` evaluate their arguments + cheapest first: `type`, `name`, `path`, `size`, `age`, `matched`, then + `duplicate`, then `content`. Content is extracted at most once per file, + and only if evaluation reaches a `content` test. +4. If a file's text cannot be extracted, `content` is false and the plan + shows a warning naming the rule that wanted it. + +### 5.5 Duplicates + +Candidates are the scanned files plus, for `(duplicate "DIR"...)`, every +regular file under those directories (symlinks skipped). Files are grouped +by size. Only groups of two or more are hashed: first the first and last +64 KiB, then SHA-256 of the whole file. Empty files are never duplicates. + +The **original** in each group is, in order of preference: a file under one +of the given DIRs, then the oldest by mtime, then the shortest name, then +the name that sorts first. `(duplicate)` is true for every other scanned +file in the group. + +## 6. Content extraction + +| Format | Method | +|---|---| +| text | known text extensions, or detected from the first 8 KiB: valid UTF-8, or UTF-16 with a BOM, and no NUL bytes | +| pdf | `pdftotext -q -enc UTF-8 FILE -`, 30 s timeout | +| docx xlsx pptx odt ods odp epub | zip plus streaming XML, Go standard library | +| html xml | tags removed, entities decoded | +| doc | `antiword`, else `catdoc` | +| xls / ppt | `xls2csv` / `catppt` (both from catdoc) | +| anything else | no content | + +External tools are optional. Each is looked up once per run; `krino check` +lists which were found. Files above `max-read` are not extracted. Tools run +without a shell, get absolute paths only (so a name starting with `-` is +never read as an option), and are killed at the timeout. + +Before matching, text and keywords are normalised the same way: case (if +`ignore`), fold (if `yes`), and runs of whitespace collapsed to one space, +so a keyword split across lines in a PDF still matches. Matching is +substring: `"acme"` matches `"acmeco"`. Words hyphenated across lines in a +PDF are not rejoined. + +## 7. Actions + +### 7.1 Chains + +A file's chain is the actions of every matching rule, in rule order, and +within a rule in the order written. The executor tracks the file's current +path: + +- `copy` leaves the file where it is; +- `rename` changes its name, `move` its directory; later steps use the new path; +- `delete` ends the chain. Steps after it appear in the plan as + "skipped: deleted by rule X". + +The plan warns when a chain moves a file more than once; that is usually a +missing `(stop)`. + +### 7.2 How each action is carried out + +- **move:** `rename(2)` when source and destination are on the same + filesystem. Otherwise: copy to a temporary file in the destination + directory, fsync, rename into place, then remove the source. If any step + fails the source is untouched and the temporary file removed. +- **copy:** to a temporary file in the destination, then rename into place. + Mode and mtime are preserved. +- **rename:** `rename(2)` within the same directory. +- **delete:** into the freedesktop.org Trash in `$XDG_DATA_HOME/Trash` + (`files/` plus a `.trashinfo` in `info/`). A file on a different + filesystem from the Trash is not trashed: the step fails with a message + suggesting `(delete permanent)` or a move. The Trash layout is the same on + Linux, FreeBSD and OpenBSD. +- **delete permanent:** `unlink(2)`. + +Before each step the executor checks that the source still exists with the +size and mtime recorded in the plan. If not, the step fails as "changed +since plan" and the rest of that file's chain is skipped. + +### 7.3 Placeholders + +| Placeholder | Value | +|---|---| +| `{name}` | the current file name | +| `{stem}` | the name without its last extension | +| `{ext}` | the last extension with its dot, e.g. `.pdf`; empty if none | +| `{1}` … `{9}` | capture groups of the first `name` test that matched in this rule | +| `{mtime:FMT}` | the file's modification time | +| `{now:FMT}` | the start of the run | +| `{{` `}}` | literal braces | + +FMT is a strftime subset: `%Y %m %d %H %M %S %j %%`. + +Captures come from the first `name` test, not inside a `not`, that was true +while evaluating this rule. Tests of equal cost keep their written order, so +"first" is well defined. `krino check` rejects a rule that uses `{N}` unless +every such `name` test in it has at least N groups, and a rule that uses +`{N}` with no `name` test at all. + +### 7.4 Conflicts + +When the target already exists: + +- `suffix` (default): use `stem_1.ext`, `stem_2.ext`, ... +- `skip`: skip this step; the chain continues from the file's current path +- `overwrite`: move the existing target to Trash first (logged, so undo + restores it), then proceed + +A `copy` whose target has identical content is skipped as "already there", +whatever the policy, so a backup rule can run every time. + +Conflicts between files in the same plan are resolved when planning, so the +plan shows final names. The executor re-checks at execution time; if the +name has to change, the log records the actual name. + +## 8. Plan, review, approval + +### 8.1 What is scanned + +- The root, recursively if `recursive yes`, to `max-depth`. +- Symlinks are never followed; symlinked files are skipped. +- `ignore` patterns use gitignore semantics: `*`, `**`, `?`, `[...]`; a + leading `/` anchors to the root; a trailing `/` matches directories only; + `!` re-includes; a pattern without `/` matches at any depth; the last + matching pattern wins. Ignored directories are not descended into. +- Always ignored: every rule `DEST` that lies inside the root, the Trash, and + the config directory. For a `DEST` with placeholders, the part before the + first placeholder is ignored: `Work/Acme/{mtime:%Y}` ignores `Work/Acme/`. +- Skipped as busy: files newer than `min-age`, or with a `busy` sibling. + +### 8.2 Display + +One directory at a time: + +``` +krino: downloads ~/downloads +266 scanned · 41 to act on · 2 warnings + + # file actions rule + 1 scan001.pdf move → Work/Acme/2026/ acme content "acme ltd" + 2 fv_123.pdf copy → ~/backup/invoices/2026/ backup content "invoice" + move → Work/Acme/2026/ acme name \bacme\b + 3 IMG_2031.JPG rename → 2026-09-01_IMG_2031.JPG photos type image + move → Photos/2026/ + 4 setup-1.2.deb DELETE permanently old-pkgs age 94d + 5 report (1).pdf trash (duplicate of Work/report.pdf) refiled + +warnings + brochure.doc content unreadable: antiword/catdoc not installed (rule acme) + big.pdf content unreadable: 120M > max-read 50M +not acted on: 3 busy · 12 ignored · 210 unmatched (-v lists them) + +[a] apply all [c] choose per file [s] skip this directory [q] quit +``` + +- Colours use the terminal's ANSI palette, only on a terminal, and never + when `NO_COLOR` is set. Permanent deletes are always marked in capitals. +- A plan taller than the terminal is shown through `$PAGER` (default + `less -FRX`) and the prompt follows when the pager exits. +- Keys are read one at a time without Enter. +- `[s]` applies nothing in this directory and moves on to the next one. + `[q]` stops krino; directories already applied in this run stay applied + and can be undone with `krino undo`. + +### 8.3 Choosing per file + +``` +[2/41] fv_123.pdf + copy → ~/backup/invoices/2026/ + move → Work/Acme/2026/ + [y] yes [n] no [a] yes to this and all remaining [d] done, apply chosen so far [q] quit, apply nothing +``` + +Approval is per file: the whole chain or none of it. + +### 8.4 Modes + +- default: plan, review, approve. +- `-y`: print the plan and apply everything. +- `-n`: print the plan and exit. `-n --json` prints it as JSON. +- If stdin is not a terminal and neither `-y` nor `-n` is given, krino + refuses rather than guess. + +## 9. Log + +Append-only, `$XDG_STATE_HOME/krino/krino.log`, one line per event, +tab-separated: + +``` +time run dir file step action status rule src dst size mtime detail +``` + +- `time`: RFC 3339 with offset. `run`: e.g. `20260911T100203-4f2a`. +- `action`: `run-start`, `mkdir`, `copy`, `move`, `rename`, `trash`, + `delete`, `displace` (a target trashed by `overwrite`), `run-end`, and the + `undo-` forms of each. +- `status`: `ok`, `failed`, `skipped`, `declined`. +- `size` and `mtime` describe the file at `dst` after the step, so undo can + tell whether it has been touched since. +- Fields are escaped: `\t`, `\n`, `\\`, and `\xNN` for control bytes and + bytes that are not valid UTF-8. Every file name round-trips exactly, and + the file stays readable with `grep` and `awk`. + +Dry runs are not logged. Files never acted on (unmatched, ignored, busy) are +not logged; declined files are. + +## 10. Undo + +- `krino log` lists recent runs: id, time, directories, counts. +- `krino undo` reverses the most recent run that has not been undone, in + every directory it touched; `krino undo RUN` a specific one. Undo runs + cannot themselves be undone. +- Undo builds a plan like any other, shown and approved the same way + (`-y` and `-n` apply). + +Reversals, last step first within each file: + +| Step | Reversal | Refused when | +|---|---|---| +| move, rename | move `dst` back to `src` | `dst` missing or changed (size/mtime), or `src` now exists | +| copy | move `dst` to Trash | `dst` missing or changed | +| trash | restore from Trash, remove the `.trashinfo` | the Trash entry is gone, or `src` now exists | +| displace | restore the displaced target from Trash | as above | +| mkdir | remove the directory if empty | not empty | +| delete permanent | none; reported as not undoable | always | + +If a reversal is refused, the earlier steps of that file's chain are not +reversed either, so no file is left half undone. Undo runs are logged like +any other run. + +## 11. Command line + +``` +krino [-y | -n] [-v] [--json] [-c FILE] [NAME...] + plan and apply every included directory, or only the named ones +krino init create ~/.config/krino with krino.conf and template.conf +krino new NAME PATH create dirs/NAME.conf from the template and include it +krino check [NAME...] validate config, list rules and available extractors +krino explain FILE evaluate every rule against FILE, show each test's result +krino log [-n N] list recent runs +krino undo [RUN] reverse a run (default: the last one) + +-y apply without asking +-n dry run: show the plan, change nothing +-v also list unmatched, ignored and busy files; full match reasons +--json with -n: the plan as JSON (format unstable before 1.0) +-c FILE use FILE instead of ~/.config/krino/krino.conf +-h, --help help +--version print "krino 0.0.1" +``` + +Exit status: 0 success, including nothing to do and everything declined; +1 one or more steps failed; 2 usage or config error; 130 interrupted. + +A config error anywhere stops the whole run before any scanning: krino never +acts on a config it only partly understood. Ctrl-C during apply finishes the +current step, logs it, and stops. A second krino on the same directory waits +for the lock, or fails immediately with `-y` (so a cron job never piles up). + +## 12. Architecture + +One binary, frontends kept thin. Everything the GUI will need lives in the +engine and returns data. + +``` +cmd/krino/ flags, subcommands, exit codes +internal/sexp/ reader: syntax tree with positions and byte offsets +internal/config/ syntax tree → typed config; validation; embedded template and defaults +internal/cond/ condition tree: compile, cost ordering, evaluation, explain trace +internal/scan/ walking, gitignore matcher, busy detection +internal/extract/ text extraction and normalisation +internal/dup/ duplicate index +internal/plan/ files × rules → chains; placeholders; conflicts; JSON +internal/apply/ executing approved steps; cross-filesystem moves +internal/trash/ freedesktop.org Trash +internal/journal/ log writer and reader, run ids, undo planning +internal/engine/ the facade: Load, Check, Plan, Apply, Explain, Runs, PlanUndo, NewDir, Init +internal/tui/ terminal rendering, pager, key prompts +``` + +Rules for the GUI to come: + +1. Every operation is an `engine` call that returns data. The CLI only + renders it. +2. `Apply` takes a plan plus the set of approved files. The CLI's `a` and `c` + and a GUI's checkboxes are the same call. +3. The reader records byte offsets, so a GUI can replace one form's text and + leave the rest of the file, comments included, untouched. +4. `Explain` returns the condition tree with each test's value. +5. Log and undo are in the engine. + +## 13. Performance + +- The walk uses `os.ReadDir` and prunes ignored directories. +- Files are evaluated by a pool of `GOMAXPROCS` workers. `pdftotext` and the + other tools run in parallel within that bound. +- Cheap tests first; content only when reached; content at most once. +- Keywords are normalised once at load time; matching is `strings.Contains` + per keyword on normalised text. Aho-Corasick only if a benchmark shows + keyword matching matters next to extraction. +- Duplicates: stat-only grouping, partial hash, then full hash. +- Measured before release on a real downloads directory of a few hundred + files, about half of them PDFs, and reported in the release notes. Goal: a + full plan with content rules in under 2 s on a 4-core laptop. `make bench` + runs the Go benchmarks on generated trees. + +## 14. Build, dependencies, release + +- Go 1.24 or newer. `CGO_ENABLED=0`: static binaries. +- Go dependencies: `golang.org/x/term` (key-at-a-time input) and + `golang.org/x/text` (Unicode normalisation for `fold`). Everything else is + the standard library, including the s-expression reader and the gitignore + matcher. +- `make` builds; Go downloads the module dependencies itself. It then checks + for the optional extractors and prints what is missing. +- `make deps` installs the extractors with the system package manager, + asking for privileges: `apt-get install poppler-utils catdoc antiword` + (Debian, Devuan), `pkg install poppler-utils antiword` (FreeBSD; catdoc is + left out because it pulls in tcl/tk), `pkg_add poppler-utils catdoc antiword` + (OpenBSD). Plain `make` never asks for privileges. +- Targets: `help build install uninstall test vet fmt lint ci bench cross + release deps install-hooks`. `install` puts the binary in + `$PREFIX/bin` (default `~/.local`), man pages in `$PREFIX/share/man`, + examples in `$PREFIX/share/doc/krino/examples`. +- `make cross`: linux/amd64, linux/arm64, freebsd/amd64, openbsd/amd64. +- `make release VERSION=0.0.1`: runs `ci` and `cross`, writes tarballs and + `SHA256SUMS` into `dist/`, and creates the annotated tag **`v0.0.1`** (the + `v` is required for `go install ...@v0.0.1`). It never pushes. +- Version stamped with `-ldflags -X main.version=...`. +- `scripts/leak-check` keeps personal data out of the repository. `make ci` + runs it, and so does the pre-commit hook that `make install-hooks` + installs. It checks the staged files against built-in patterns (the + current user's home directory, email addresses) and against a private + pattern list whose path is set with `git config krino.leakpatterns FILE`. + The list lives outside the repository, so the secrets it names are never + published either. Without it, only the built-in patterns run. +- `/local/` is ignored by git: a place for personal configs and test data + next to the code. + +Repository contents for 0.0.1: `README.md` (60-second quickstart), +`LICENSE`, `CHANGELOG.md`, `Makefile`, `go.mod`, `cmd/`, `internal/`, `scripts/`, +`man/krino.1`, `man/krino.conf.5`, `docs/design.md`, +`docs/sexp-primer.md`, `examples/` (by type, invoices by tax number with +placeholder numbers, screenshots by date, cleaning up old installers), +`testdata/`. + +## 15. Testing + +- **Unit tests** per package, table-driven. +- **sexp:** positions and offsets (every list's offsets point at its own + parentheses), error positions, and a fuzz target for the reader. +- **gitignore:** an oracle test that builds trees and compares every + path against `git check-ignore --no-index` (skipped if git is absent). +- **Conditions:** truth tables; a property test that random condition trees + give the same result with and without cost reordering. +- **Extraction:** small fixtures in `testdata/` for each format; PDF and + legacy-format tests skip when the tool is missing. +- **Plan, apply, undo:** in temporary directories: build a tree, plan, + apply, check; undo; check the tree is identical to the start (paths, + contents, modes, mtimes). A cross-filesystem move test runs when a tmpfs is + available on a different device, and skips otherwise. +- **Trash:** with `XDG_DATA_HOME` pointed at a temporary directory. +- **CLI:** golden files for plan rendering without colour; exit codes. +- **Parity check before release:** run `krino -n` beside the prototype's + dry run over the same real directory, and explain every difference. The + rules and the results stay outside the repository. + +`make ci` is the gate: gofmt, vet, tests, the leak check, man page lint. + +## 16. Lessons from the prototype + +krino replaces a Python prototype that sorted a downloads folder by +keyword. Its defects shaped these decisions: + +| Prototype | krino | +|---|---| +| filename keywords were substrings, so a short keyword matched inside unrelated words (`"art"` matched `start.pdf`) | name tests are regexes, e.g. `\bart\b` | +| `.doc` and `.xls` content was never read (the libraries do not support those formats) | antiword or catdoc, or a warning | +| code files were listed as supported, but their content was never read | any text file is read | +| the rules lived in the repository beside the code | rules live in `~/.config/krino`, never in the repository (§14 leak check) | +| it installed Python packages with `pip --break-system-packages` | a static binary; system packages via `make deps` | + +## 17. Decisions and why + +| Decision | Why | +|---|---| +| Config in one central place | target directories stay clean; one place to back up; the tool never sorts its own config | +| Every matching rule runs, `(stop)` to end | chains like copy-then-move need it; `stop` gives first-match behaviour where wanted | +| Tests see the original file | the plan is fully known before approval, and files can be evaluated in parallel | +| s-expressions, not TOML, YAML or INI | conditions are the core of the config; in sexp they are structure rather than a string in a second language. No nested quoting (TOML's `'''` around single-quoted regexes), no precedence, actions in written order. YAML turns `*` and `!` in ignore patterns into syntax; INI has no standard escaping | +| `pdftotext` rather than a Go PDF library | best text quality of the open tools, fast, packaged on all three systems | +| Delete to Trash by default | recoverable, and undo can restore it | +| A new repository rather than the prototype's | the prototype's history holds personal data; the prototype keeps working until krino reaches parity | +| Personal configuration never enters the repository | rules hold private data; the leak check enforces it on every commit and in `make ci` | + +## 18. Settled before implementation + +1. **Hosting: local only for now.** The module path is `krino`. When a + remote is chosen, `go mod edit -module ` and one `sed` over the + `"krino/internal/...` imports rename it; nothing else depends on it. + `go install ...@v0.0.1` works only after that. +2. **License: GPL-3.0-or-later.** Full text in `LICENSE`; every source file + carries `// SPDX-License-Identifier: GPL-3.0-or-later`. + +## Appendix A: type groups + +| Group | Extensions | +|---|---| +| `image` | jpg jpeg png gif webp bmp tif tiff heic heif avif svg ico raw cr2 nef arw dng | +| `video` | mp4 mkv webm mov avi m4v mpg mpeg wmv flv 3gp | +| `audio` | mp3 flac ogg opus m4a aac wav wma aiff | +| `archive` | zip tar gz tgz bz2 tbz2 xz txz zst 7z rar lz lzma cpio | +| `document` | pdf doc docx odt rtf txt md tex | +| `spreadsheet` | xls xlsx ods csv tsv | +| `presentation` | ppt pptx odp | +| `ebook` | epub mobi azw azw3 fb2 djvu | +| `code` | go c h cpp hpp py sh js ts rs java rb pl lua html css json yaml yml toml xml sql | +| `text` | txt md log csv tsv json yaml yml toml xml ini conf | +| `package` | deb rpm apk appimage exe msi flatpak snap | +| `font` | ttf otf woff woff2 | + +Groups overlap; a file can belong to several. diff --git a/docs/sexp-primer.md b/docs/sexp-primer.md new file mode 100644 index 0000000..c72c1e1 --- /dev/null +++ b/docs/sexp-primer.md @@ -0,0 +1,300 @@ +# S-expressions: a primer for krino + +krino's configuration is written in **s-expressions**, the notation Lisp has +used since 1958. You do not need to know any Lisp to write a krino config. This +page teaches exactly the part of the notation krino uses, where it comes from, +and how to read a rule at a glance. It ends with exercises and a reading list. + +## 1. Two kinds of thing + +Everything in an s-expression is one of two things. + +An **atom** is a single word or value: + +```lisp +pdf ; a symbol: a bare word +30d ; also a symbol, krino reads it as "30 days" +"acme ltd" ; a string: anything in double quotes +``` + +A **list** is a pair of parentheses holding atoms or other lists, separated by +spaces: + +```lisp +(type pdf docx) +(move "Work/2026") +(when (type pdf) (size > 10M)) +``` + +That is the whole grammar. There are no commas, no semicolons between items, +no operators with special spelling, no indentation rules. Whitespace and line +breaks only separate things; you can lay a list out on one line or twenty. + +A `;` starts a comment that runs to the end of the line. By convention `;;` +begins a comment on its own line and `;` a comment after code. + +## 2. The first element says what to do + +In every list, the **first element names the operation** and the rest are +what it operates on: + +```lisp +(type pdf docx) ; operation: type arguments: pdf, docx +(move "Photos") ; operation: move argument: "Photos" +(and A B C) ; operation: and arguments: A, B, C +``` + +This is **prefix notation**: the operator comes before its operands. Ordinary +arithmetic is *infix* (`3 + 4`, operator in the middle). Prefix writes it +`(+ 3 4)`. + +Because the parentheses mark exactly where each list ends, an operator can +take any number of arguments. `(+ 1 2 3 4)` adds four numbers; `(and A B C)` +requires three conditions; `(type pdf docx odt)` accepts three extensions. +Infix cannot do that without repeating the operator: `1 + 2 + 3 + 4`. + +## 3. Where prefix notation comes from + +The Polish logician **Jan Łukasiewicz** introduced prefix notation in 1924, to +write logic without any parentheses at all: "I came upon the idea of a +parenthesis-free notation in 1924." It became known as **Polish notation**. + +The trick works when every operator takes a fixed number of operands. Then +the order alone determines the grouping: + +| Infix | Polish (prefix) | Reverse Polish (postfix) | +|-------------------|-----------------|--------------------------| +| `3 + 4` | `+ 3 4` | `3 4 +` | +| `(3 + 4) × 5` | `× + 3 4 5` | `3 4 + 5 ×` | +| `3 + (4 × 5)` | `+ 3 × 4 5` | `3 4 5 × +` | + +**Reverse Polish notation** puts the operator *after* its operands. It is how +stack calculators and the Unix `dc` calculator work. You can try it now: + +```sh +echo '3 4 + 5 * p' | dc # (3 + 4) × 5, prints 35 +echo '3 4 5 * + p' | dc # 3 + (4 × 5), prints 23 +``` + +(`p` tells `dc` to print the top of its stack.) + +Lisp's s-expressions are Polish notation **with the parentheses kept**. That +costs a few brackets and buys two things: operators may take any number of +arguments, and you can always see at a glance where an expression ends. + +## 4. Nesting + +A list can contain lists, to any depth. Read nested lists from the outside +in: the outer operation first, then each argument in turn. + +```lisp +(or (type image) (size > 10M)) +``` + +"**or** of two things: the file is an image; the file is bigger than 10 MB." + +```lisp +(and (type document) + (or (content "acme ltd") (name "\bacme\b")) + (not (name "^draft"))) +``` + +"**and** of three things: +1. the file is a document; +2. **or** of two things: its text mentions *acme ltd*; its name contains the word *acme*; +3. **not**: its name starts with *draft*." + +There is no precedence to remember. In infix, `A and B or C` could mean +`(A and B) or C` or `A and (B or C)`, and you have to know the rule. In an +s-expression you must write one or the other: + +```lisp +(or (and A B) C) ; (A and B) or C +(and A (or B C)) ; A and (B or C) +``` + +## 5. What a krino config looks like + +A krino config file is a sequence of lists. Each top-level list is a +**form**, and the first word of the form says what it sets: + +```lisp +;; ~/.config/krino/dirs/downloads.conf +(path "~/downloads") +(ignore "*.part" "*.aria2" ".*") + +(rule "acme" + (when (type document) + (or (content "acme ltd" "0000000000") + (name "\bacme\b"))) + (move "Work/Acme/{mtime:%Y}") + (stop)) +``` + +`(rule "acme" ...)` is a form whose first argument is the rule's name and +whose remaining arguments are more lists: `when` holds the condition, +`move` and `stop` are actions. The whole rule is itself just a list. + +Two krino conventions to know: + +- **`when` with several conditions means all of them.** The rule above + could have been written `(when (and (type document) (or ...)))`; the `and` + is implied. You need `and` only inside an `or` or a `not`. +- **Tests that take several arguments mean "any of them".** `(type pdf docx)` + is true for a PDF *or* a DOCX. `(content "a" "b")` is true if the text + contains *a* or *b*. For "all of", combine tests: `(content "a") (content "b")`. + +### Symbols or strings? + +Bare words (`pdf`, `yes`, `strict`, `>`, `30d`) are **symbols**. Paths, +keywords, rule names and regular expressions go in **double quotes**. If a +value could contain a space, a parenthesis, a `;` or a `"`, quote it. + +Inside a string, a backslash escapes only `"` and `\`. Every other backslash +is kept as written, so regular expressions need no doubling: + +```lisp +(name "\bacme\b") ; the regex \bacme\b, exactly as written +(content "say \"hello\"") ; the text: say "hello" +``` + +## 6. Translating ordinary logic + +| You mean | You write | +|--------------------------------------------------------|----------------------------------------------------| +| PDFs | `(type pdf)` | +| PDFs or Word files | `(type pdf docx)` | +| PDFs over 10 MB | `(type pdf) (size > 10M)` | +| images or anything over 1 GB | `(or (type image) (size > 1G))` | +| mentions "invoice" but not "draft" | `(content "invoice") (not (content "draft"))` | +| name starts with `IMG_`, in strict case | `(name "^IMG_")` plus `(case strict)` on the rule | +| a document, and either the name or the text says acme | `(type document) (or (name "acme") (content "acme"))` | + +(The left column of conditions all go inside `(when ...)`.) + +## 7. Common mistakes + +**Unbalanced parentheses.** Every `(` needs a `)`. krino reports the line +where an unclosed list *opened*, which is usually the rule that is broken: + +``` +downloads.conf:12:1: "(" never closed: (rule "acme" ...) +``` + +Count from the inside out, or let your editor do it (see below). + +**`not` takes exactly one condition.** To negate several, wrap them: +`(not (or A B))` means "neither A nor B". + +**A bare word where a string is needed.** `(move Work/Acme)` is a symbol with +a slash in it, which krino rejects for a path; write `(move "Work/Acme")`. + +**Forgetting the implied `and` is only at the top of `when`.** Inside `or`, +each argument is a separate alternative: +`(or (type pdf) (size > 1M))` is "PDF, or big", never "big PDF". + +## 8. Editor help + +Any editor with a Lisp mode matches parentheses and indents s-expressions. +krino's generated files start with lines that switch it on: + +```lisp +;; -*- mode: lisp -*- +;; vim: set ft=lisp : +``` + +In **vim**: `%` jumps between matching parentheses; `:set showmatch` flashes +the partner of each `)` you type; `=` re-indents a selection using Lisp rules. +In **Emacs**: `C-M-f` / `C-M-b` move over a whole list; `C-M-q` re-indents it. + +## 9. Exercises + +1. Write `(2 + 3) × 4` as an s-expression, in Polish notation, and as a `dc` + command. Check the `dc` one in your shell. +2. Write the condition: "videos or audio files older than a year". +3. Write the condition: "spreadsheets that mention *budget* but not *draft*". +4. Find the mistake: `(when (and (type pdf) (content "invoice"))` +5. Do these mean the same? `(or (and A B) C)` and `(and A (or B C))` +6. Write a rule named `screenshots` that moves PNG files whose name starts + with `Screenshot` into `Pictures/Screenshots`, and stops. + +### Answers + +1. `(* (+ 2 3) 4)`; `* + 2 3 4`; `echo '2 3 + 4 * p' | dc` prints `20`. +2. `(when (type video audio) (age > 365d))`. One `type` with two groups + already means "either". +3. `(when (type spreadsheet) (content "budget") (not (content "draft")))` +4. One `)` is missing at the end: the `when` list is never closed. Count: + `(when` opens 1, `(and` 2, `(type` 3 closes to 2, `(content` 3 closes to + 2, and the final `)` closes `and`, leaving `when` open. +5. No. Take A false, C true: the first is true (because of C), the second is + false (because A is false). +6. ```lisp + (rule "screenshots" + (when (type png) (name "^Screenshot")) + (move "Pictures/Screenshots") + (stop)) + ``` + +## 10. Further reading + +All links checked on 2026-09-11. + +**Start here** + +- Paul Graham, *The Roots of Lisp* (2002). A short essay that rebuilds Lisp + from seven primitive operators; the clearest explanation of why + s-expressions are both code and data. + +- Wikipedia, *S-expression* and *Polish notation*. + , + +- *Learn X in Y minutes: Scheme*. One page of syntax. + + +**Learn to think in s-expressions** + +- Daniel P. Friedman and Matthias Felleisen, *The Little Schemer*, 4th ed., + MIT Press, 1995, ISBN 978-0-262-56099-3. Written entirely as questions and + answers; the gentlest route. +- Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, Shriram + Krishnamurthi, *How to Design Programs*, 2nd ed. A free beginner course. + +- Harold Abelson and Gerald Jay Sussman, *Structure and Interpretation of + Computer Programs*, 2nd ed. Chapter 1 is enough for this purpose. Free HTML + edition: +- Robert J. Chassell, *An Introduction to Programming in Emacs Lisp*. Also + available as `info eintr` where Emacs is installed. + +- Peter Seibel, *Practical Common Lisp* (2005). Chapter 3 builds a small + database out of s-expressions, close in spirit to a krino config. + +- Daniel Higginbotham, *Clojure for the Brave and True*. A modern Lisp. + + +**Try it interactively** + +- GNU Guile (Scheme). Debian: `guile-3.0`; OpenBSD: `guile3`; FreeBSD: + `pkg search guile`. +- Racket. Debian: `racket`; OpenBSD: `racket-minimal`. + +- `dc(1)` for reverse Polish notation: `man dc`. + +**Build one** + +The fastest way to understand a notation is to write a parser for it. + +- Daniel Holden, *Build Your Own Lisp*. A Lisp in C, parser included. + +- *mal: Make a Lisp*. A step-by-step guide with implementations in dozens of + languages, Go among them. + +**Original sources** + +- John McCarthy, "Recursive Functions of Symbolic Expressions and Their + Computation by Machine, Part I", *Communications of the ACM*, 1960. The + paper that defined s-expressions. + +- R. Rivest and D. Eastlake 3rd, RFC 9804, *Simple Public Key Infrastructure + (SPKI) S-Expressions* (2025). S-expressions specified as a data format. + -- cgit v1.3