VersionControl:PRs/maybelle-config/292

maybelle-config#292: Branch tests on Jenkins, on an agent with no secrets

State	🟣 merged 2026-11-09 01:28:13
Worked on by	magent, re, justin
Moods	#ramp
Deploy	redeploy maybelle (then the hunter mirror needs nothing)
Branch	magent/branch-tests → production, by magent-cryptograss
Opened · last pushed	2026-11-09 01:13:30 · 2026-11-09 01:22:29

Branch tests on Jenkins, without branch code ever running beside Jenkins' secrets (Plan, section 11, as revised after re's review; Justin, 9 Nov: on maybelle, Jenkins for the console). magenta's `ci/test` is magenta#443.

**The job, `branch-tests`:** one fixed pipeline, written here, never a branch's Jenkinsfile. re's point: a Jenkinsfile chooses its own node, and its Groovy runs on the controller.
- It takes REPO, BRANCH and COMMIT, checked against strict patterns before anything runs.
- It clones exactly that commit from GitHub, anonymously, on the `branches` agent.
- It runs the repository's own `ci/test` inside `sh`. Branch code runs there and nowhere else.
- No `ci/test` means NOT_BUILT, with a line in the console saying so.
- Consoles stay public.

**The agent:** its own container (`cryptograss-jenkins-branches`: Python, Node, git, ssh-keygen, a compiler), with no secrets and no `jenkins_home`.
- **Node mode EXCLUSIVE**, so only `branch-tests` lands there and nothing trusted ever does.
- **`ci/test` runs as `builder`, never as the agent's own user** (re's review). The agent's secret belongs to the agent's user, so code running as that user could print it into a public console, and anyone could then connect a fake `branches` agent.
  - `run-ci` is the one thing the agent's user may sudo, with no arguments.
  - Before the run, it hands the workspace to `builder`, then runs `ci/test` with a clean environment and a fresh home.
  - Afterwards it kills every `builder` process, removes what `builder` left outside the workspace, and hands the workspace back.
  - The node has one executor, so that cleanup can't kill another build.
  - Previewed in the image: `builder` can't read the secret, list the agent's home or sudo; the exit status comes through; a leftover process and file are gone after; `run-ci` refuses arguments and anywhere but a build's workspace.
- **Its secret:** Jenkins derives it from the node name. It's read inside Jenkins' own container, as admin, so the password never leaves it, and written 0400 for the agent's user. `agent.jar` reads it with `-secret @file` (checked against the image's agent.jar).
- **Its network, `branches-net`:** `internal: true` plus `inhibit_ipv4`, so the host has no address on that bridge. Previewed on hunter in throwaway containers:
  - without `inhibit_ipv4`, the host's sshd answered through the gateway;
  - with it, the gateway, 10.0.0.2 (memory-lane, githouse, postgres), 10.0.0.4 and 169.254.169.254 were all unreachable.
- **Its only way out is `branches-proxy`** (tinyproxy), which allows GitHub, npm and PyPI by name and refuses everything else, IP addresses included. Previewed: github and pypi gave 200; example.com and 10.0.0.2 were refused.
- **Every deploy checks from inside the agent** that nothing of ours answers, and that the proxy refuses `CONNECT 10.0.0.2:3000`, a plain `GET http://10.0.0.2:3000/` and `CONNECT github.com:22` (previewed: 403 each). The deploy fails if anything gets through. `branches-net` also says `enable_ipv6: false` outright.

**githouse:** after a push reaches GitHub, it asks Jenkins to test exactly that commit, as the reporter, who can only read and start builds. Where Jenkins is and the reporter's password live in `/etc/githouse` files, since hooks get none of the container's environment. A failure is logged and never fails the push.

**`ci/test` for maybelle-config:** what `.github/workflows/tests.yml` runs, except the pinning service. That is githouse, maybelle scripts, hunter, render_diff and the recovery gate. It passes here.

**Not checked:**
- The JCasC `inbound:` launcher key: older Jenkins called it `jnlp:`. If it's wrong, the deploy's JCasC load says so.
- The whole thing against a real Jenkins: the first deploy is that test.
- Still to do from section 11: credentials scoped to the trusted jobs, the controller's environment emptied, and the reporter's Build permission narrowed to `branch-tests` and the deploy-report jobs (re). Its password now lives on githouse too, so that one comes first. Branch code can't reach either from the agent, so those are defence in depth, next.

Tests: githouse 49 (3 new: a forwarded push asks for its commit; a deletion or a refused forward doesn't; not set up, or Jenkins down, is logged and never raised). render_diff: 376 identical, 5 new.

Deploy: redeploy maybelle (then the hunter mirror needs nothing)
Worked on by: magent, re, justin, in #ramp