diff options
Diffstat (limited to 'straper/README.md')
| -rw-r--r-- | straper/README.md | 269 |
1 files changed, 0 insertions, 269 deletions
diff --git a/straper/README.md b/straper/README.md deleted file mode 100644 index d6923d5..0000000 --- a/straper/README.md +++ /dev/null @@ -1,269 +0,0 @@ -# sanctum / labunix rebuild toolkit - -A modular disaster-recovery toolkit for rebuilding a Debian 13 server (`sanctum`) in a controlled, validated order. - -``` -capture → lint → install → restore → doctor -``` - -Rebuild time: weeks → hours. All risky changes are explicit, reversible, and tested. - ---- - -## Toolkit layout - -``` -~/.local/bin/straper/ -├── capture-full.sh Capture live server state into db/ -├── lint-db.sh Validate DB shape before restore -├── install-base.sh Install packages and baseline -├── restore-configs.sh Restore config categories from DB -├── doctor.sh Read-only health and readiness checks -├── rebuild.conf.example Optional config overrides -├── .gitignore Excludes db/secret/ from git -├── lib/ -│ └── common.sh Shared helpers, restore_path(), report() -└── db/ Rebuild database (auto-created by capture-full.sh) - ├── public/ Unencrypted inventory — git-tracked - └── secret/ Root-only (chmod 700) — never committed -``` - -The toolkit is **self-contained** — `capture-full.sh` writes the DB to `db/` inside the toolkit directory. No external paths needed. - -Runtime artifacts: `/var/log/labunix-rebuild/` and `/var/lib/labunix-rebuild/`. - ---- - -## Roles - -| Role | Use for | Behavior | -|------|---------|----------| -| `lab` | VM testing, dry runs | Skips DNS/firewall/network, no identity restores | -| `hardware` | Real machine migration | Full adaptive restore, services started | -| `replacement` | True disaster recovery | All restores including private identities | - ---- - -## Canonical workflow - -### Step 1 — Capture (on source host) - -```bash -sudo ~/.local/bin/straper/capture-full.sh -``` - -Writes to `~/.local/bin/straper/db/public/` and `db/secret/`. Run after any significant change. - -### Step 2 — Lint - -Always lint before restore. Do not proceed if lint fails. - -```bash -sudo ~/.local/bin/straper/lint-db.sh -``` - -Catches: nested duplicate roots, overlapping captures, backup junk, duplicate canonical files. - -### Step 3 — Transfer to target - -```bash -rsync -av ~/.local/bin/straper/ lukasz@TARGET:~/rebuild-test/sanctum-rebuild-toolkit/ -``` - -Both toolkit and DB are transferred in one rsync — they live in the same directory. - -### Step 4 — Install base - -```bash -cd ~/rebuild-test/sanctum-rebuild-toolkit -sudo bash ./install-base.sh --role hardware --profile core -``` - -Installs packages (including Grafana repo setup, Docker, dns-root-data), sets up baseline, stops conflicting services. Does **not** restore any config. - -### Step 5 — Restore categories - -Restore in blocks. Run `doctor.sh` after each block. - -**Safe first block:** -```bash -sudo bash ./restore-configs.sh \ - --role hardware --yes \ - --category system-basics \ - --category users \ - --category ssh -``` - -**Service block:** -```bash -sudo bash ./restore-configs.sh \ - --role hardware --yes \ - --category nginx \ - --category mariadb \ - --category postfix \ - --category prosody \ - --category docker \ - --category monitoring \ - --category tor \ - --category i2pd -``` - -**TLS:** -```bash -sudo bash ./restore-configs.sh \ - --role hardware --yes \ - --category tls -``` - -**Risky block (run last):** -```bash -sudo bash ./restore-configs.sh \ - --role hardware \ - --category dns \ - --category firewall \ - --category network -``` - -### Step 6 — Verify - -```bash -sudo bash ./doctor.sh --role hardware -``` - ---- - -## Smart adaptive restore - -`dns`, `firewall`, and `network` adapt to the target machine rather than blindly copying files. - -### DNS (`restore_dns`) - -1. Scans live interfaces, IPs, WireGuard ifaces, Docker bridges, port 53 owner -2. Adapts `dnsmasq.conf` and `dnsmasq.d/*` — strips non-live IPs, comments out absent `interface=` lines -3. Installs `dns-root-data` if missing, runs `unbound-helper` to seed trust anchor -4. Disables DNSSEC validator for bootstrap (re-enable after system stable) -5. Disables `systemd-resolved` stub if it conflicts with dnsmasq -6. Fetches `root.hints` from internic.net **after** DNS is confirmed working -7. Starts unbound → dnsmasq in correct order, verifies each is listening -8. Writes `/etc/resolv.conf` only after resolution is confirmed -9. Prints precise manual task list - -WireGuard and Docker rules are preserved as dormant comments — activate automatically when those interfaces come up. - -### Firewall (`restore_firewall`) - -1. Scans live interfaces -2. Comments out rules referencing absent interfaces (`wg0`, `enx*`, `br-*`/`docker0`) -3. Validates with `nft -c`, loads with `nft -f`, enables and starts `nftables.service` -4. Docker bridge rules collapsed to single manual task - -### Network (`restore_network`) - -Restores `/etc/network`, `systemd-network`, `nsswitch.conf`, `hosts.allow/deny`. Refreshes postfix chroot after nsswitch restore. Skipped in `lab` role. - ---- - -## Validated restore order - -``` -system-basics → users → ssh -nginx → mariadb → postfix → prosody → docker → monitoring → tor → i2pd → tls -dns → firewall → network -``` - -DNS before firewall. Network last. TLS before DNS (nginx needs certs to start). - ---- - -## Validated doctor baseline (hardware role) - -``` -Summary: ok=27 warn=2 fail=0 manual=1 -``` - -Expected permanent warnings: `kvm` (VM only), DNS chain intentional. - ---- - -## Restore categories - -| Category | Risk | Notes | -|----------|------|-------| -| `system-basics` | low | hostname, hosts, locale, timezone | -| `users` | low | sudoers with ownership fix, shells | -| `ssh` | low | sshd_config; host keys in hardware/replacement only | -| `nginx` | low | lab: default site only; enables service | -| `mariadb` | low | /etc/mysql tree with normalization | -| `postfix` | low | main.cf, master.cf; enables and starts service | -| `prosody` | low | tree + cert permission normalization | -| `docker` | low | daemon.json; adds user to docker group | -| `monitoring` | low | prometheus, loki, grafana, alloy trees | -| `tor` | low | torrc; /var/lib/tor only in replacement role | -| `i2pd` | low | /etc/i2pd tree; /var/lib/i2pd only in replacement role | -| `tls` | low | letsencrypt certs from secret DB; reloads nginx | -| `dns` | **risky** | smart adaptive — see above | -| `firewall` | **risky** | smart adaptive — see above | -| `network` | **risky** | skipped in lab | - ---- - -## Known lessons - -**Metadata matters.** Content restore alone is not enough — owner, group, mode must be normalized. `restore_path()` handles this. - -**DB shape matters.** A polluted DB produces bad restores even with correct logic. Always lint first. - -**Tree restores are overlay-style.** Use a fresh VM or clean the destination before retesting. - -**resolv.conf is sacred.** Never write it until a local resolver is confirmed listening. `install-base.sh` always writes a static file (breaking the systemd-resolved symlink) at bootstrap time. - -**DNSSEC on Debian 13.** `unbound-anchor` is not shipped — use `dns-root-data` package + `unbound-helper`. Disable validator module for initial bring-up; re-enable after stable. - -**Service start order.** unbound must be listening before dnsmasq starts. - -**nftables interface references.** Rules referencing absent interfaces must be commented out or nftables refuses to load entirely. - -**Docker on Debian.** `docker.io` ships v1 compose (`docker-compose`), not the v2 plugin (`docker compose`). Use `docker-compose` or install `docker-compose-plugin` separately. - ---- - -## Public vs secret DB tiers - -`db/public/` — unencrypted, git-tracked. Safe for remote backup. - -`db/secret/` — root-only (`chmod 700`), excluded by `.gitignore`. Protected at rest by ZFS-on-LUKS. Contains SSH keys, TLS private keys, WireGuard keys, LUKS headers, service secrets. - ---- - -## Quick reference - -```bash -# Capture (self-contained — writes to db/ in toolkit dir) -sudo ~/.local/bin/straper/capture-full.sh - -# Lint -sudo ~/.local/bin/straper/lint-db.sh - -# Transfer toolkit + DB to target in one rsync -rsync -av ~/.local/bin/straper/ lukasz@TARGET:~/rebuild-test/sanctum-rebuild-toolkit/ - -# Install -sudo bash ./install-base.sh --role hardware --profile core - -# Restore (non-interactive) -sudo bash ./restore-configs.sh --role hardware --yes --category dns - -# Verify -sudo bash ./doctor.sh --role hardware - -# List categories -sudo bash ./restore-configs.sh --list-categories -``` - ---- - -## Status - -All 15 restore categories validated including risky (`dns`, `firewall`, `network`). -Validated doctor baseline: `ok=27 warn=2 fail=0`. -Suitable for: structured lab testing, staged hardware migration, disciplined disaster-recovery rehearsal. |