Hydra tutorial series — Part 7: CI and deployment
Parts 5 and 6 ran Hydra on your own machine. This part zooms out to how Hydra runs unattended, triggered by Codeberg/Forgejo Actions, with container images in the Codeberg registry, scoped credentials and token rotation. It lays out the three deployment models and marks honestly what is documented and what is infrastructure-only.
In part 5 you started Hydra by hand: build the images, run the supervisor, label an issue. That is the right way to learn Hydra, but it is not how it runs day to day. In production nobody runs hydra-supervisor.sh from a terminal. Instead the forge itself triggers the pipeline when an issue is labelled, pulls pre-built images from a registry, and runs each stage as a CI job. This part explains that path, and the three deployment models behind it.
Same pipeline, different trigger
Nothing about the pipeline changes between local and CI. The personas, the labels, the gates, the no-loop rule from part 2: all identical. What changes is who pulls the trigger and where the work runs:
| Local (part 5) | CI / production (this part) | |
|---|---|---|
| Trigger | you run hydra-label.sh / hydra-supervisor.sh | the forge fires on issues: labeled |
| Where stages run | containers on your machine | CI runner jobs |
| Images | you docker build them | pulled from the Codeberg registry |
| Credentials | your local secrets/ | forge org secrets and variables |
So everything you learned about labels and the state machine still applies. You are just handing the steering wheel to the forge.
Trigger: Codeberg/Forgejo Actions
Hydra's canonical CI lives in .forgejo/workflows/. Codeberg runs Forgejo Actions, which speak the same YAML dialect as GitHub Actions. There are three workflows:
-
hydra-build.ymlfireson: issues: [labeled]. It checks the label against the trigger name and, on a match, runs the builder stage. The match honours the label prefix from part 5:on: issues: types: [labeled] jobs: build: if: github.event.label.name == format('{0}{1}ready-to-build', vars.HYDRA_LABEL_PREFIX, vars.HYDRA_LABEL_PREFIX != '' && '-' || '') -
hydra-review.ymlfires on pull-request events for Hydra's feature branches, running the code-review and security-review stages. -
hydra-image.ymlbuilds the four persona images and pushes them to the registry, so the other two workflows have something to pull.
The single entry point is unchanged from part 1: a tracking issue labelled ready-to-build (or <prefix>-ready-to-build). Locally a polling daemon notices that label. In CI the forge's event system notices it. Same contract, different plumbing.
Images: the Codeberg Container Registry
In a local run you build the four images yourself. In CI that would waste minutes on every job, so the images are built once by hydra-image.yml, pushed to the Codeberg Container Registry, then pulled by the build and review jobs. The naming is driven by two variables:
HYDRA_IMAGE_PREFIX="${HYDRA_IMAGE_PREFIX:-codeberg.org/conduction/hydra}"
HYDRA_IMAGE_TAG="${HYDRA_IMAGE_TAG:-latest}"
So the reviewer image resolves to codeberg.org/conduction/hydra-reviewer:latest, and the same for builder, security and applier. Pinning HYDRA_IMAGE_TAG to a digest or a dated tag is how you freeze a known-good pipeline version instead of always tracking latest.
Credentials and token rotation
Production Hydra never uses a shared org-admin token. That is a hard security rule. Two credential sets live in git-ignored secrets/credentials.json (org secrets in CI):
{
"claude_tokens": [ // multiple accounts, tried in order
{ "name": "account-1", "token": "..." },
{ "name": "account-2", "token": "..." }
],
"git_tokens": { // one scoped token PER persona
"builder": "...",
"reviewer": "...",
"security": "..."
}
}
Two things matter here:
- Per-persona git tokens. The builder, reviewer and security personas each push with their own scoped token. Their commits and review comments are attributable, and their permissions are minimal. No persona can act as another.
- Claude token rotation. When a run hits a rate limit, the container-launch helper rotates to the next Claude token in the list (
run_container_with_fallbackplusscripts/lib/github-rotate.sh) instead of failing the stage. Several Claude Max accounts inclaude_tokenskeep a busy fleet moving.
The three deployment models
Hydra's docs/deployment-models.md describes three ways to run it, in increasing operational weight.
Troubleshooting CI runs
Common CI-specific snags
Check the HYDRA_LABEL_PREFIX variable on the forge matches the label you applied. The if: guard in hydra-build.yml builds the expected label name from that variable. A mismatch means the event is ignored silently.
The registry tag is wrong or the image was never built. Confirm hydra-image.yml ran and pushed the image at codeberg.org/conduction/hydra-<role> with the expected tag, and that the runner has pull access to the registry.
Verify all accounts are present in claude_tokens and that the secret is actually mounted in the job. Rotation only helps if there is more than one valid token to rotate to.
A persona is using the wrong scoped git token. Each of builder, reviewer and security needs its own entry in git_tokens. A shared token breaks attribution and the least-privilege model.
Test yourself
1. What actually changes between a local run (part 5) and a CI run, and what stays the same?
Hint
Think about the trigger and the runtime versus the pipeline itself.
Answer
What changes: the trigger (a forge event on issues: labeled instead of your local supervisor daemon), where stages run (CI runner jobs instead of local containers), where images come from (registry pull instead of local docker build), and where credentials live (org secrets instead of local secrets/).
What stays the same: the entire pipeline. Personas, labels, the state machine, the 22 gates, the no-loop rule. The contract is still one tracking issue labelled ready-to-build.
2. Why does each persona have its own git token instead of one shared token?
Hint
Two reasons: who did what, and how much each can do.
Answer
Attribution (each persona's commits and comments are traceable to that persona) and least privilege (each token carries only the permissions that role needs, and no persona can act as another). A shared org-admin token is explicitly forbidden.
3. Which deployment model is the everyday production path, and which one lacks a written operator runbook?
Hint
One runs on Actions runners. One runs on Kubernetes.
Answer
Model 2 (CI runner, Forgejo or GitHub Actions) is the everyday production path for the fleet. Model 3 (Kubernetes and ArgoCD) has real manifests but no step-by-step operator runbook yet, so you read the manifests directly. Model 1 (local) is parts 5 and 6.
Done — the whole series
That is the full Hydra picture, from concept to production. Where to from here: