Sovereign Stack

plocic-infra is the single source of truth for everything that runs behind plocic.dev. One small Hetzner VM, one docker compose up, and every side project lives there — back end and front end: Przepisnik, Sumo, a crash-report ingest, self-hosted analytics and real-user web vitals, search, image resizing, a mailer, a private npm registry, a private image registry, monitoring, and the web apps themselves — all behind one Caddy reverse proxy with automatic TLS. The page you’re reading is served from it. The box’s /opt/app is a checkout of this repo; push to main and it deploys itself.

It exists because the alternative was a stack of per-project SaaS bills — a managed Postgres here, a hosted PocketBase there, a transactional-email plan, a search-as-a-service tier — each one small, all of them together a monthly tax on the privilege of having shipped something. This replaces the lot with a single predictable VM and a repo I own end to end.

Every new project used to mean another invoice. Now it means another twelve-line block in docker-compose.yml.

The shape of it

Each PocketBase is a single Go binary against SQLite, so the marginal cost of another project is essentially zero. They share the expensive parts — TLS, search, image processing, mail, monitoring — and Caddy fans the subdomains out to internal containers that never touch the public internet.

Hetzner VM (CX23 · Ubuntu 26.04 · Helsinki)        Hetzner Storage Box
│
├─ Caddy :80/:443/:443udp   ── auto Let's Encrypt, HTTP/3
│   │
│   │  ── the web tier (prod + preview lane each) ──
│   ├─ www.plocic.dev        ─→ plocic-web      (Astro SSR, node)
│   ├─ sumo.plocic.dev       ─→ sumo-web        (Next.js SSR, standalone)
│   ├─ przepisnik.app        ─→ przepisnik-web  (SPA on caddy:alpine)
│   ├─ preview.*             ─→ …-preview       (same three, :preview tag)
│   │
│   │  ── the data tier ──
│   ├─ pocket.przepisnik.app ─→ przepisnik (PocketBase)
│   ├─ pocket.sumo.plocic.dev ─→ sumo       (PocketBase) + sumo-worker
│   ├─ report.plocic.dev      ─→ errors     (PocketBase, opt-in reports)
│   ├─ vitals.plocic.dev      ─→ analytics  (PocketBase, web-vitals ingest)
│   ├─ stats.plocic.dev       ─→ umami      (cookieless traffic · Postgres)
│   ├─ view.*.plocic.dev      ─→ read-only vitals + report dashboards
│   │
│   │  ── shared services ──
│   ├─ registry.plocic.dev    ─→ registry    (private OCI images, htpasswd)
│   ├─ search.przepisnik.app  ─→ meilisearch
│   ├─ img.przepisnik.app     ─→ imgproxy
│   ├─ mail.plocic.dev        ─→ mailer
│   ├─ npm.plocic.dev         ─→ verdaccio
│   └─ monitor.plocic.dev     ─→ beszel + dozzle
│
├─ weekly · registry retention ── archive old prod images ─────────────┐
└─ nightly · sqlite .backup + secrets ─→ borg (encrypted, deduped) ─→ ┘

Everything but Caddy lives on the internal docker network. The firewall opens exactly three ports — SSH from my IP, 80/443 from everywhere — and denies the rest. PocketBase’s 8090 is never published; the only way in is through the proxy.

Then it grew a web tier

The box started as a backend stack — the frontends still lived on Vercel, which made the whole “stop renting” argument a bit convenient. That’s finished now. All three frontends run here: Sumo as a Next.js standalone Node server, Przepisnik as a static SPA on a tiny caddy:alpine, and this site as an Astro SSR Node server. The edge Caddy fronts all of them; none of them is exposed directly.

The delivery model is the part worth stealing. This repo does not build anyone’s app — each project’s own CI builds its image and pushes it to a private OCI registry at registry.plocic.dev, and this repo owns only the slot and the Caddy route. Every app gets two long-lived environments and no per-branch previews:

• push to main → CI pushes :preview → the preview service redeploys → preview.plocic.dev
• push a v* tag → CI pushes :prod → the prod service redeploys → the real domain

Each build also pushes an immutable tag alongside the moving one — main-<sha7> for previews, the git tag for prod — so a rollback is a re-point, not a rebuild. That’s a nice property and a slow disk leak, which is why the registry now prunes itself on a weekly systemd timer: previews keep only the current image, prod keeps the current plus the previous one for instant rollback, and older prod images are archived to the Storage Box before they’re deleted. The safety detail that bit me first: a registry deletes by digest, not by tag, and the moving tag and the immutable tag from one build point at the same digest — so the pruner has to compute the set of digests referenced by everything it keeps before it deletes anything, or “delete the old v1.6.0 tag” quietly takes :prod with it.

Deploys that know what changed

The part I’m proudest of is the GitHub Action. It doesn’t blindly docker compose up on every push — it diffs the repo against the host by content hash (CI rewrites mtimes on every checkout, so timestamps lie) and decides the smallest correct action:

• changed docker-compose.yml / a Dockerfile → rebuild and recreate
• changed the Caddyfile only → live caddy reload, no container restart
• changed the report ingest → restart just that one container

Before anything touches the host, a validation gate runs offline: docker compose config, caddy validate, shellcheck, and a completeness check that fails the build if an env var is used in compose but undocumented in .env.example. Then it snapshots the current config and image IDs, syncs, health-checks eight public endpoints over real DNS+TLS, and auto-rolls-back — config and image tags only, never data — if any of them returns a 5xx.

Backups that assume the worst

The data layer is SQLite, which is mid-write at any given moment, so the nightly job doesn’t just rsync pb_data. It takes a consistent online copy of each SQLite database with .backup, pg_dumps the one Postgres in the stack (Umami’s), bundles them with uploaded files and every secret needed to come back from nothing, and pushes the lot to a Hetzner Storage Box via Borg — encrypted, compressed, deduplicated — on a systemd timer with jitter so it never lands on a round minute. Retention is 7 daily / 4 weekly / 6 monthly.

Derived data — the Meilisearch index, the monitoring history — is deliberately not backed up. It’s computable, so it’s rebuilt from source after a restore by a bootstrap script. Backups stay small and honest about what’s actually source-of-truth.

Disaster recovery is a feature, not a hope

The repo is explicit that two things, together, can resurrect the entire stack on a blank server: the git repo (all code, compose, routing, Dockerfiles) and the Borg archive + passphrase (all data and secrets). Neither alone is enough — the git repo can’t conjure your data, the Borg archive can’t run without the compose — and that separation is the point. The README walks the full path: provision a box, git clone to /opt/app, borg extract, drop the secrets back, docker compose up --build, rebuild the search index. Time-to-resurrection is measured in minutes, and it’s been written down before it was ever needed.

Owning the analytics, too

The last third-party dependency on the public site was the way it measured itself — @vercel/analytics for traffic, @vercel/speed-insights for Core Web Vitals. Both are gone now, replaced by two services on this box, because “stop renting your backend” rang hollow while a vendor still watched every visitor. Traffic runs through Umami — cookieless, MIT, no cross-site tracking — at stats.plocic.dev, backed by the single Postgres in the whole stack. Real-user vitals — LCP, CLS, INP, FCP, TTFB — land in an infra-owned PocketBase at vitals.plocic.dev, the same trick as the error ingest: the schema, hooks, and hardening migrations are committed here and mounted read-only, so vanilla PocketBase plus this repo is the service.

The collector is a ~2 KB deferred keepalive beacon that fires once, when the page is hidden — late enough that it can’t move a Lighthouse lab score. That’s deliberate: measuring real users shouldn’t tax the synthetic number, and it definitely shouldn’t tax the real one. Both services ship inert behind a docker-compose profile — a normal deploy doesn’t even start them — and Umami’s Postgres folds into the nightly Borg run.

On top of both sits a pair of read-only dashboards, view.vitals and view.report, built in the site’s own design language instead of a vendor’s console: p75 rollups with a plain-language explainer of what the percentile means, interactive chart tooltips, and Lighthouse-style good / needs-improvement / poor rating shapes.

The shared services, briefly

• mailer — a 162-line, zero-dependency Node service that renders branded transactional email per project and dispatches over an HTTPS API (Hetzner blocks outbound SMTP). One mailer, many brands, per-key rate limits.
• report — an opt-in crash/error ingest on its own PocketBase. The API key ships in the frontend and is not the security boundary; a strict server-side schema gate (field allowlist, type checks, length caps) and per-IP rate limiting are. A leaked key can’t store junk.
• verdaccio — a private npm registry at npm.plocic.dev so @plocic/* packages (like the @plocic/report client) live on my own box, with public deps cached from npmjs.
• imgproxy + meilisearch — on-the-fly AVIF/WebP resizing and typo-tolerant search, shared across whichever projects want them.

The service I deleted

For a while the box also ran ollama, serving a local model to Sumo’s editorial desk. It was the most exciting thing on the machine, and it’s gone: drop ollama — unused AI experiment, frees ~17GB on the box.

This is a CPU VPS, and that turns out to be the whole story. The models big enough to be reliable — 35b and up — are bound here by both CPU and RAM. The models that fit comfortably, the 9–11b tier, weren’t reliable enough to rephrase a paragraph that had already been written correctly for them, which is the only job the editorial pipeline gives a model. Too slow to use, or too small to trust; there’s no third door on hardware like this.

The honest fix wasn’t more RAM, it was arithmetic. A GPU VPS is $500+/month; the desk needs about twenty generations a month; an EU-hosted gpt-oss-120b at Scaleway does the same work for roughly four cents a month, with ten times the parameters of anything that could ever have run here. Sovereignty was never about the silicon being in my hallway — it’s about jurisdiction and who can reach the data — and that survives a metered EU API just fine. The full story is in the notebook: the model may not source a fact.

A stack you own is one you’re allowed to subtract from. That’s most of the point — and the corollary is that self-hosting is a tool, not an identity. It earns its place per workload, and this workload didn’t buy it one.