aboutsummaryrefslogtreecommitdiff
path: root/internal
Commit message (Collapse)AuthorAgeFilesLines
* a file's name is folded once, not once per name testLukasz Kasprzak46 hours4-1/+31
| | | | | | | | | | Folding is the expensive half of a name test on a name with diacritics, and every name test of every rule folded the same name again: on Polish names it was most of the matching work. The per-file facts memoise it, which is where one file's work belongs - the object is per file and per goroutine, so no lock. 4000 Polish names, twelve rules with name tests: 0.33s -> 0.13s
* max-read bounds what a file becomes, not only what is readLukasz Kasprzak46 hours3-8/+59
| | | | | | | | | | | | | | | | | | | | | max-read gates on file size before reading, then the text was read whole and decoded: a file that is not valid UTF-8 decodes one byte per code point and doubles, and normalising and folding copy that again per set of options, with GOMAXPROCS files in flight. Twelve 40 MB files reached 3.3 GB - enough to put a laptop into the OOM killer, with no attacker involved, just a few big .log or .csv files. Two bounds. The decoded text is cut to max-read at a rune boundary, so the ceiling means what a reader takes it to mean. And extraction of files at or above 4 MiB is rationed to two at a time, since holding several large texts at once is what multiplies the ceiling; smaller files, which is nearly all of them, are untouched. twelve 40 MB files: peak RSS 3294 MB -> 728 MB, wall 27s -> 46s two thousand small files: 0.05s both ways The wall-clock cost falls entirely on large files needing extraction, and buys a program that finishes instead of being killed.
* {now} is the start of the run, as the spec always saidLukasz Kasprzak46 hours3-4/+71
| | | | | | | | | | | | | The clock was read once per directory, so a run of several directories stamped several different times - and a review that took a few seconds could put one run's files into two folders, or at midnight two dates. The spec (§7.3) says "the start of the run"; krino.conf(5) documented the behaviour rather than the intent, so the two contradicted each other. The session now carries the run's clock and every directory plans with it. Engine.Plan keeps its meaning for a caller with no session of its own; Session.Plan passes the run's own start.
* a symlink in the sorted directory no longer redirects a stepLukasz Kasprzak46 hours5-5/+136
| | | | | | | | | | | | | | | | | | | | | | | | | | Placeholders were already stopped from sending a file out of the directory a rule named. A symlink is a name too, and the directory krino sorts is by the threat model's own premise a place the internet writes into: a link named after a rule's destination sent moves and copies anywhere, and under (on-conflict overwrite) trashed a file OUTSIDE the sorted directory - while the plan showed the in-tree text and the run reported success. A step whose destination passes through a symlink at or below the directory being sorted now fails. A destination the configuration names outside it - ~/docs on another disk - is the user's own arrangement and is followed as before; both cases have a test. End to end, the review's scenario (Out -> ~/secret, overwrite): before: 1 applied, the user's file replaced and trashed after: 0 applied 1 failed, the file untouched, nothing trashed Two bookkeeping bugs in MkdirAllTracked went with it: a dangling symlink read as a missing directory and was then recorded as one krino had created - undo would have unlinked a link krino never made - and a directory created by someone else between the check and the mkdir was recorded the same way.
* a duplicate test in an exclude protects like one in a ruleLukasz Kasprzak46 hours2-9/+57
| | | | | | | | | | | | The scopes that drive "no rule deletes a file krino found to be a duplicate" were collected from rules only. A directory whose duplicate tests lived in (exclude ...) forms had no scopes at all, so the protection never engaged: krino explain said "yes duplicate" and the next rule permanently deleted every copy. The README's promise was false in that shape, and the spec's wording permitted it. What matters is what krino knows, not which form taught it. The spec and krino.conf(5) now say so too.
* a chain that deletes itself still gives back the file it displacedLukasz Kasprzak46 hours4-2/+206
| | | | | | | | | | | | | | | | | | | | | | | | (on-conflict overwrite) trashes the file in the way; §7.4 promises undo restores it. Walking a file's log entries stopped dead at a permanent delete, so the displace written earlier in the same chain was never reached: the user's file stayed in the Trash, the refusal named only the file they did not care about, and krino log called the run undone. The displaced file is a different file, so it is offered as its own entry in the undo plan, keyed by its own path - the deleted file stays refused, since nothing of it can come back, and the copy or move that preceded the delete stays unreversed too (undoing a copy whose original was then deleted would destroy the last remaining copy). The accounting matched: every reversible step of a deleted file was subtracted, its displace included, so the run read (undone). Only what genuinely cannot come back is subtracted now. End to end, the scenario from the review: the only copy of a file is displaced by an incoming one that is then permanently deleted. before: archive/ empty, "(undone)", nothing offered after: archive/a.pdf restored, run reads partly undone
* a file trashed to make room is logged the moment it is trashedLukasz Kasprzak47 hours3-26/+140
| | | | | | | | | | | | | | | | | | | The displace was carried out first and logged only when the whole step finished - for a copy or a cross-device move, the entire data transfer later. A process killed in that window left the user's file in the Trash with nothing recording it: krino log said "nothing applied" and undo offered nothing. It is its own action with its own line (spec §9), so it is now written the moment trash.Put returns, through a hook ChainLogged calls before the step that needed the name begins. A displace that cannot be logged fails the step rather than compounding an unrecorded destructive act with a second one. Verified by killing krino -9 mid-copy with 600 MB in flight: before: krino log "nothing applied", 0 displace lines after: krino log "1 displaced", 1 displace line
* the suffix search continues instead of starting again at _1Lukasz Kasprzak47 hours4-13/+89
| | | | | | | | | | | | | | | Every file renamed onto one name probed stem_1, stem_2, ... from the beginning, so N files cost N^2/2 Exists calls - a test here counts 1890 of them for 60 files. The search now continues from the highest suffix already tried for that stem. Within one plan that is the same answer: the taken set only grows while a plan is built and the disk is not being written to, so a suffix taken once stays taken. Proved rather than argued - with same_1 and same_3 already on disk and same_2 free, both versions put a file in the gap, and the two plans are byte-identical. 1500 files renamed to one name: 2.63s -> 0.05s
* a content class is worked out once, not once per copyLukasz Kasprzak47 hours2-5/+98
| | | | | | | | | | | | | | | | | | Lookup walked the whole size group for every file in it, so N copies of one file cost N walks of an N-member group, each taking the index's lock at every step - which is why more workers made it slower rather than faster. Every member of a class elects the same original (the invariant identicalTo already documents and a test already pins), so the class is memoised on the first walk and every later member is a map lookup. Measured over identical files, invented data, same machine: 500 files 0.17s -> 0.12s 2000 files 1.89s -> 0.58s and the plans are byte-identical once the sandbox path is normalised. The test counts walks: one per content class, not one per file.
* the directory lock is the kernel's, not a pid we believeLukasz Kasprzak47 hours2-100/+208
| | | | | | | | | | | | | | | | | | | | | | | | | | flock(2) on the lock file's descriptor replaces "write my pid, and decide whether the pid in the file is still alive". The kernel drops the lock when the process ends, however it ends, so there is no stale krino lock to detect and no takeover to race over. What that fixes: - Two runs that both judged a lock stale could remove and recreate it and both believe they held it. Remove-then-create cannot be made atomic; there is nothing to make atomic now. Pinned by a test with eight callers over twenty rounds. - A pid reused after a crash made the lock live for ever, and the message named neither the file nor the pid, so there was nothing to act on. The message now names both. - Signal(0) reads EPERM as "not running", so a lock held by another user was taken over. There is no such judgement left to get wrong. A run that waits for a held lock now says so first. Waiting is what the spec asks for, but the wait has no timeout, and in silence it is indistinguishable from a hang - I spent two minutes on one myself today, waiting on a lock the window was holding. Tests that faked a held lock by writing a file now hold a real one.
* the module path is git.labunix.xyz/krinoLukasz Kasprzak48 hours50-105/+105
| | | | | | | | | | | So that go install can find it. cgit serves no go-import meta tag, so nginx answers a ?go-get=1 request for /NAME with one pointing at https://git.labunix.xyz/NAME.git; the alternative was carrying a .git suffix through every import line forever. One sed over the imports, both go.mod files, and the dependency gate's whitelist. Nothing else depends on the path. go install works from the next tag, the first release whose go.mod carries it.
* comments that explain the code, not how it was writtenLukasz Kasprzak2 days43-704/+644
| | | | | | | | | | | | | | | | | | About 340 comments cited the development process: task and plan numbers, fix waves, rulings, reviewers, and the author in the third person with a date. None of that exists outside the work itself, so to a reader it pointed at nothing. Each one now states the engineering reason it was standing in front of; where a comment was provenance and nothing else, it is gone. References to docs/design.md and docs/gui-design.md by section stay: both ship with the repository. The design documents lose their amendment diaries - CHANGELOG.md is that record - and the GUI's says plainly that the window has gone further than the document. Only comments changed. Every .go file was parsed and its code printed with comments stripped, before and after: the two hashes are identical across all 175 files.
* gui: name the other copy of a duplicate, and offer to keep this one insteadLukasz Kasprzak2 days3-0/+54
|
* gui: syntax colours, a file preview, and a settings windowLukasz Kasprzak3 days1-0/+5
|
* gui: Rules tab - the file as text, checked as you type, tested and savedLukasz Kasprzak3 days1-0/+6
|
* gui: History and undo tab; each plan and undo is its own runLukasz Kasprzak3 days2-2/+55
|
* unsaved text for another included directory is not an error; two keys for ↵Lukasz Kasprzak3 days2-19/+76
| | | | one file are
* milestone 1 review: claims span the run, explain's chain is opt-in and its ↵Lukasz Kasprzak3 days9-30/+366
| | | | own, overrides keyed by clean path, splice and enum guards
* config prints and splices one formLukasz Kasprzak3 days5-2/+268
|
* undo runs through the same sessionLukasz Kasprzak3 days2-10/+89
|
* the engine owns a run: lock, log, run id, claimsLukasz Kasprzak3 days2-0/+266
|
* explain reports captures and the chain a file would getLukasz Kasprzak3 days3-18/+108
|
* config and engine can load with unsaved textLukasz Kasprzak3 days4-3/+73
|
* check refuses an empty time format; krino.conf(5) corrected from the re-auditLukasz Kasprzak4 days2-0/+5
|
* check refuses placeholders that could never expandLukasz Kasprzak4 days3-0/+57
|
* an undecided (stop) rule ends the search for that fileLukasz Kasprzak4 days2-0/+52
|
* a failed duplicate lookup is unknown, like unreadable contentLukasz Kasprzak4 days4-5/+96
|
* docs: GUI design draft; code comments no longer name the ownerLukasz Kasprzak4 days3-7/+6
|
* a permanently deleted file's steps do not keep a run partly undoneLukasz Kasprzak4 days2-2/+47
|
* a rule's duplicate check failure shortens the path tooLukasz Kasprzak4 days2-8/+74
|
* (matched) is unknown after an undecided rule, so a catch-all leaves an ↵Lukasz Kasprzak4 days6-14/+82
| | | | unreadable file alone
* captures keep the combining marks of a decomposed name; test sources escape ↵Lukasz Kasprzak4 days3-7/+20
| | | | invisible characters
* explain walks the directory for a duplicate test in an exclude tooLukasz Kasprzak4 days2-3/+44
|
* partial reads never cached; extractor version 3 discards 0.0.7 cachesLukasz Kasprzak4 days2-1/+32
|
* an earlier directory's applied results stay claimed for later onesLukasz Kasprzak4 days1-0/+6
|
* tests read real documents from LibreOffice and pandoc; antiword's ↵Lukasz Kasprzak5 days10-0/+57
| | | | short-document limit documented
* krino log marks a run partly undone while reversible steps remainLukasz Kasprzak5 days2-5/+81
|
* tests: apply error exits 1 and w stops krino through the real command; undo ↵Lukasz Kasprzak5 days1-0/+53
| | | | projection occupied half
* build: VERSION checked for build and install, cross names without v, ↵Lukasz Kasprzak5 days2-0/+51
| | | | dependency gate covers tests and BSDs, tarball install, examples and extractor list tested
* main excludes checked with the defaults' case and fold; Latin-1 decoding ↵Lukasz Kasprzak5 days4-6/+35
| | | | memory; hardlink election documented
* output: wrap by terminal columns, names cannot fake step lines, -v lists ↵Lukasz Kasprzak5 days1-2/+26
| | | | unscanned destinations
* duplicate warnings share one formatLukasz Kasprzak5 days3-9/+75
|
* --json carries exclusions and matching warningsLukasz Kasprzak5 days2-13/+42
|
* literal braces are text, not placeholders, for containment and walk exclusionLukasz Kasprzak5 days7-25/+69
|
* explain: shows a duplicate's skipped delete, leaves the cache alone, walks ↵Lukasz Kasprzak5 days2-3/+42
| | | | only for duplicate tests
* content tests are three-valued: unknown when unreadable or read in partLukasz Kasprzak5 days7-47/+227
|
* captures keep the name's diacriticsLukasz Kasprzak5 days7-8/+194
|
* tests run on OpenBSD; a tool's error message survives one large write; lock ↵Lukasz Kasprzak5 days4-11/+33
| | | | pids beyond 32 bits name no process
* text that turns binary further on has no content, like an imageLukasz Kasprzak5 days4-19/+16
|
* plan 10 re-check: cut run column, damaged undo run, emptied directory ↵v0.0.7Lukasz Kasprzak5 days9-9/+175
| | | | cleanup, text turning binary, explain flags, interrupt docs