From d7da4b09f7775276231d0241cfe2700d247728ee Mon Sep 17 00:00:00 2001 From: Lukasz Kasprzak Date: Thu, 30 Jul 2026 08:56:34 +0200 Subject: sources: store the snapshot as one compressed archive Replace the ~1600 raw per-provider files (24 MB) with a single solid sources/snapshot.tar.gz (6 MB); manifest.tsv + README stay uncompressed for browsing. The extracted per-provider dirs are now git-ignored; snapshot-sources.sh gains pack/unpack, and the README documents the unpack -> refresh -> pack cycle. Distribution-ready: one archive, not a file tree. --- sources/README.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) (limited to 'sources/README.md') diff --git a/sources/README.md b/sources/README.md index b177c3d..f746d29 100644 --- a/sources/README.md +++ b/sources/README.md @@ -7,9 +7,18 @@ re-audited offline** if those sites change, move, or disappear — the project's 25-year-longevity aim. Reading references are facts, not creative works; this is a provenance cache, not redistribution of anyone's edition. -Refresh or extend with `sh scripts/snapshot-sources.sh [provider ...]` -(idempotent: existing non-empty files are skipped). Every file is recorded in -`manifest.tsv` (path, url, sha256, bytes). First archived **2026-07-29**. +The data ships as a single compressed archive, **`sources/snapshot.tar.gz`** +(~6 MB); `manifest.tsv` (path, url, sha256, bytes) and this README stay +uncompressed for browsing. First archived **2026-07-29**. + +```sh +# unpack the archived data (into git-ignored per-provider dirs): +sh scripts/snapshot-sources.sh unpack # or: tar xzf sources/snapshot.tar.gz -C sources +# refresh/extend from the live sites (idempotent — skips files already present): +sh scripts/snapshot-sources.sh # all four providers +# re-roll the archive after refreshing: +sh scripts/snapshot-sources.sh pack # or: tar czf sources/snapshot.tar.gz -C sources +``` ## What's here -- cgit v1.3