Building hal0, part 1: fifteen tags in ten days

Between July 4th and July 13th I tagged hal0 fifteen times. v0.9.0 on the 4th, 0.9.7 on the 11th, 0.9.8 two days after that, with a scatter of point releases in between. Nobody was asking me to move that fast — there was no launch date, no customer waiting on a feature. I was moving fast because I kept finding the seam between “works on my box” and “is actually a platform,” and every time I found one I wanted to close it before I lost the thread.

hal0 is a home AI inference appliance. The pitch, if you want the one-liner: stop running models from a chat tab. Point it at a Linux box — I built it against a Ryzen AI Max+ 395 with 128 GB of unified memory, but it doesn’t insist on that — and it turns the box into a private, OpenAI-compatible inference surface. Chat, completions, embeddings, reranking, transcription, speech and image generation all answer on one :8080/v1 endpoint. No per-token bill, because there’s no vendor in the loop.

None of that is the interesting part yet. The interesting part is what had to be true architecturally for the pitch to survive contact with a real, messy, single-box deployment.

1. One container per workload, no shared daemon

The obvious way to build this is a single inference process that loads whatever model the request asks for. I did not build that, on purpose. Every inference workload in hal0 runs as its own podman container, managed by its own hal0-slot@<name>.service systemd unit. hal0-api on :8080 is the only thing that talks to the outside world — it owns the slot state machines, dispatches /v1/* requests to whichever port the right slot is listening on, and serves the dashboard. There is no shared inference daemon sitting in the middle deciding what to load next.

I keep saying this out loud in the README because people keep assuming hal0 is “another llama-server wrapper,” and it explicitly isn’t. A wrapper shares one process’s memory space and one process’s failure domain across every model you ever load. That’s fine until it isn’t — until one workload’s crash takes the whole box down, or one model’s memory leak starves the model next to it, or you want two different runtimes (llama.cpp for chat, a totally different stack for image generation) living in the same address space and pretending that’s normal.

The slot model punts all of that to the kernel and to systemd, which are both better at process isolation than I will ever be at hand-rolling it. A slot is a name, a port, a model, and a state. hal0-api renders that state into a podman Quadlet file — a .container unit that systemd itself expands into a real service. I don’t write ExecStart=podman run anywhere; the generator does it, which means I get systemd’s dependency graph and restart semantics for free instead of reimplementing them badly:

The shape is real; the angle-bracketed values are whatever the provider’s plan resolved to for that slot. WantedBy=hal0.target is the load-bearing line — hal0.target is the boot-enable anchor every slot depends on, and if it is missing or installed-but-not-enabled, slots that looked perfectly healthy before a reboot silently stay down after one. That failure mode was common enough that hal0 doctor now has a dedicated check for it.

Restarting a wedged model doesn’t touch anything else on the box. Killing hal0-api for a redeploy doesn’t kill the models that were already warm — a graceful shutdown drains in-flight pulls instead of SIGKILLing them, which by 0.9.8 had already saved me from corrupting more than one download mid-restart. And when I want to add a new workload type — image generation joined as img, the brain steward joined as brain — it’s a new template, not a new code path bolted onto an existing one.

2. The capability system is the interface, not the container

Slots are the mechanism. Capabilities are what an operator actually thinks in. hal0 ships ten seeded slots out of the box — agent, brain, coder, embed, flm, img, qwen3tts, rerank, tts, utility — and the installer’s whole job is to seed those, ask where models should live, optionally take a HuggingFace token, and start the API. No separate first-run wizard. The curl | bash line is the setup.

What I actually spent the ten days on wasn’t the container plumbing — that had mostly settled by v0.9.0. It was making the capability layer trustworthy on top of it: which slot can serve a chat completion right now, which one is warming, which one just died. 0.9.6 added FLMProvider.verify_embed, a one-shot /v1/embeddings readiness sentinel, specifically because the NPU trio’s warm→ready gate wasn’t modality-aware — an FLM slot running embed- or STT-primary with chat disabled would wedge on a chat probe it was never going to satisfy instead of promoting on the readiness signal that actually applied to it. That’s the kind of bug that only shows up once you’ve decided the capability, not the container, is the unit an operator reasons about. If the abstraction is honest, “is embed ready” has to mean something real.

3. What the Lemonade cut actually cut

The CHANGELOG for this project starts at v0.2.0 on purpose, tagged “the Lemonade migration cut.” Everything before that lived somewhere else, under a different name, with a different launcher underneath it. I’m not going to relitigate that migration here, but it’s worth saying plainly: the slot model is downstream of that decision, not incidental to it. Once the launcher layer was mine to own, “one process per model” stopped being a constraint I was working around and became a choice I could actually make on purpose — and once I made it, most of the roughness in the following months was just working out its consequences.

0.9.7 was the release where those consequences paid off the most visibly. I called it “the steward release” in the CHANGELOG, and the name earned itself: the dashboard’s agent chat graduated from a side panel into a real control surface, running the full hal0-admin MCP catalog — which grew from 28 tools to 74, replacing the slide-out’s old hardcoded ~25-tool list — under a per-persona tool policy that pauses a turn on a gated tool for inline approve/deny instead of just… doing it. agent and brain slots replaced chat as the default anchor, seeded on every startup rather than only on a fresh install. And hal0 update verification actually started working again — release signing now dual-emits a Sigstore bundle with an embedded Rekor timestamp, because the old detached signature had no trusted timestamp and cosign verification was failing 100% of the time on curl | bash installs run more than about ten minutes after the release was cut. That one had been silently broken since v0.9.2.

A slot is a name, a port, a model, and a state. Everything else hal0 does is either rendering that into a container spec, or lying to you about it — and I’d rather find the lies early.

4. What was still duct tape at 0.9.8

I want to be honest about this because the next two posts in this series are, structurally, about fixing what’s in this section. By 0.9.8, Turnstone had landed as a second heavyweight bundled agent alongside Hermes, coexisting through a relaxed single-pick mechanism that let two agent frameworks share a box without stepping on each other. hal0-rocmfpx became the unified default runner image for AMD GPUs — one image (Vulkan/RADV + HIP) replacing the per-lane toolboxes, with an automatic slot migration on update. The PyPI package got renamed to hal0ai — a genuinely boring change, the kind you only notice when it goes wrong.

None of that means the platform was solid. 0.9.7.3, a robustness release one day after 0.9.7, existed entirely because fresh installs kept failing on boxes that weren’t shaped exactly like mine: hardened umasks, non-root operation, a host running docker instead of podman. The installer had to learn to auto-provision its own Python and Node rather than assuming they were already there. The self-hosted Honcho memory stack — which existed as a swappable alternative to Hindsight at the time — needed its own standup path debugged against pgvector dimension mismatches and AppArmor-in-LXC weirdness before it would come up cleanly next to Hindsight. And on the security side, 0.9.8 shipped a fix for a real bug: in unified-bank memory mode, one agent could delete another agent’s visibility:private memory by guessing its document id, because delete hadn’t been enforcing the same fail-closed ACL that read and search already had.

That last one is the shape of bug that recurs through this whole project, and it’s worth naming now because it’s the thread the rest of this series pulls on: a system that looks consistent — read is protected, so surely delete is too — can still have a gap that only a specific kind of attention finds. At 0.9.8, I was finding those gaps by hand, one report at a time, on a single box I controlled completely. That does not survive other people’s hardware. Finding out exactly how it doesn’t survive is what August was for.