summaryrefslogtreecommitdiff
path: root/internal/extract
Commit message (Collapse)AuthorAgeFilesLines
* max-read bounds what a file becomes, not only what is readLukasz Kasprzak2 days2-8/+33
| | | | | | | | | | | | | | | | | | | | | max-read gates on file size before reading, then the text was read whole and decoded: a file that is not valid UTF-8 decodes one byte per code point and doubles, and normalising and folding copy that again per set of options, with GOMAXPROCS files in flight. Twelve 40 MB files reached 3.3 GB - enough to put a laptop into the OOM killer, with no attacker involved, just a few big .log or .csv files. Two bounds. The decoded text is cut to max-read at a rune boundary, so the ceiling means what a reader takes it to mean. And extraction of files at or above 4 MiB is rationed to two at a time, since holding several large texts at once is what multiplies the ceiling; smaller files, which is nearly all of them, are untouched. twelve 40 MB files: peak RSS 3294 MB -> 728 MB, wall 27s -> 46s two thousand small files: 0.05s both ways The wall-clock cost falls entirely on large files needing extraction, and buys a program that finishes instead of being killed.
* comments that explain the code, not how it was writtenLukasz Kasprzak2 days4-40/+38
| | | | | | | | | | | | | | | | | | About 340 comments cited the development process: task and plan numbers, fix waves, rulings, reviewers, and the author in the third person with a date. None of that exists outside the work itself, so to a reader it pointed at nothing. Each one now states the engineering reason it was standing in front of; where a comment was provenance and nothing else, it is gone. References to docs/design.md and docs/gui-design.md by section stay: both ship with the repository. The design documents lose their amendment diaries - CHANGELOG.md is that record - and the GUI's says plainly that the window has gone further than the document. Only comments changed. Every .go file was parsed and its code printed with comments stripped, before and after: the two hashes are identical across all 175 files.
* docs: GUI design draft; code comments no longer name the ownerLukasz Kasprzak4 days2-2/+2
|
* partial reads never cached; extractor version 3 discards 0.0.7 cachesLukasz Kasprzak5 days1-1/+4
|
* tests read real documents from LibreOffice and pandoc; antiword's ↵Lukasz Kasprzak5 days10-0/+57
| | | | short-document limit documented
* build: VERSION checked for build and install, cross names without v, ↵Lukasz Kasprzak5 days1-0/+14
| | | | dependency gate covers tests and BSDs, tarball install, examples and extractor list tested
* main excludes checked with the defaults' case and fold; Latin-1 decoding ↵Lukasz Kasprzak5 days2-5/+23
| | | | memory; hardlink election documented
* content tests are three-valued: unknown when unreadable or read in partLukasz Kasprzak5 days3-4/+14
|
* tests run on OpenBSD; a tool's error message survives one large write; lock ↵Lukasz Kasprzak5 days2-10/+16
| | | | pids beyond 32 bits name no process
* text that turns binary further on has no content, like an imageLukasz Kasprzak5 days3-13/+8
|
* plan 10 re-check: cut run column, damaged undo run, emptied directory ↵v0.0.7Lukasz Kasprzak5 days3-6/+14
| | | | cleanup, text turning binary, explain flags, interrupt docs
* plan 9: undo review matches review, --min-age validated, future mtimes, rule ↵Lukasz Kasprzak5 days2-1/+19
| | | | names, conflict enum, dependency gate, absolute tool paths
* plan 8: fuzz decoders; fold ẞ and invalid UTF-8 correctly, refuse ↵Lukasz Kasprzak5 days1-0/+56
| | | | non-UTF-8 paths in krino new
* krino: 0.0.5 — keyword cache, t and d in reviewv0.0.5Lukasz Kasprzak5 days2-0/+50
|
* krino: matching — scan, ignore, conditions, extraction, duplicates, ↵Lukasz Kasprzak8 days7-0/+1579
explain, dry run
generated by cgit v1.3 (git 2.53.0) at 2026-09-19 19:52:50 +0000