Cryptograss:Moods/ramp/todo: Difference between revisions
From PickiPedia: A knowledge base of bluegrass, old time psychedelic jams, and other public domain music
Jump to navigationJump to search
#293 deployed halfway; #294 (via update-page on MediaWiki MCP Server) |
branch-tests startup race; #295 (via update-page on MediaWiki MCP Server) |
||
| Line 220: | Line 220: | ||
kind: merge | kind: merge | ||
link: https://github.com/cryptograss/maybelle-config/pull/294 | link: https://github.com/cryptograss/maybelle-config/pull/294 | ||
note: " | note: "Merged and deployed 01:56Z: the agent is online, the isolation check passed. branch-tests was on disk but never loaded (a startup race); fix: #295." | ||
done: true | |||
- task: "Merge maybelle-config#295 (reload Configuration as Code after Jenkins starts; check every job exists) and redeploy maybelle" | |||
who: justin | |||
kind: merge | |||
link: https://github.com/cryptograss/maybelle-config/pull/295 | |||
note: "Then branch-tests exists, and githouse's next push gets tested." | |||
done: true | done: true | ||
- task: "Section 11, the rest: the reporter's Build narrowed to branch-tests and the deploy-report jobs (first: its password is on githouse now); credentials into a folder only the trusted jobs see; the controller's environment emptied (env_file included)" | - task: "Section 11, the rest: the reporter's Build narrowed to branch-tests and the deploy-report jobs (first: its password is on githouse now); credentials into a folder only the trusted jobs see; the controller's environment emptied (env_file included)" | ||
Revision as of 02:23, 9 November 2026
- ☐maybelle-config#246 (read-only GitHub token for memory-lane): merge or closeOptional now that #408 is live. Justin, 6 Nov: no appetite for another token. Close it unless the asks starve again.
- ☐
- ☐Decide Plan sections 8 and 9 (merging without a PR; a page per piece of work): verify, or push backre reviews the design first. Open questions: SSH signature per merge (CLI, then passkey on the phone) versus trusting a signed-in device; who may merge.
- ☐Re-verify Plan section 10 (rev 15035: one tier, phone merges everywhere; the rule that no merge can take away your way to stop it) and the runbook (rev 15038)Also Cryptograss:Runbooks/A key is compromised.
- ☐Check which account maybelle_github_token (Jenkins' github-token) belongs to and its scopes, without printing it: pickipedia and arthel merges run code in Jenkins with itIf it can administer the org, a merge there can change the org and its deploy keys (Plan, section 10, open).
- ☐hunter's ssh router refuses a banned person's key; a redeploy prints any change in people, keys, githouse_mergers or MOOD_ADMINS (scripts/render_diff.py)Plan section 10, alongside.
- ☐
- ☐memory-lane tells the Moods what githouse merged, from /srv/git/.merges (read-only mount)
- ☐hunter's deploy builds filled its disk (7 Nov 02:37Z: the mirror's fetches failed with 'No space left on device' until it recovered by 03:42Z). Docker holds 32 GB of reclaimable images and 22 GB of build cache: prune after building, in the hunter playbookA maybelle-config PR, not a hand prune: servers change through deploy logic.
- ☐Watches, the rest: the merged event line in the Mood from githouse (today only the webhook writes it); a deploy watch (server, commit), which needs deploys to stamp the commit they deployedPlan, section 4. The merge half is #415. Deploy stamping is the prerequisite nothing does yet.
- ☐Deploys clone from githouse's copy on maybelle instead of GitHub (Jenkinsfiles, deploy scripts)re's point 9: its own PR, after githouse has run a few days.
- ☐Jenkins audit, re's open items: a policy on silent skips (pickipedia node/lua checks), pickipedia's chain-data warning that deploys stale silently, NVM/pnpm fetched from the network in every pipelineFrom re's test map, 30 Oct. The mtime guard (#244) and the alarms (#402, #275) are done.
Done (38)
- ☑Review the reworked maybelle-config#250 (githouse on maybelle, mirror on hunter; your ten points of 2 Nov folded in)Reviewed 6 Nov 04:44Z: sound, two bugs found (the hooks lacked the deploy key; the port check was inverted), both fixed on the branch by magent from re's description.
- ☑Merge maybelle-config#250 and redeploy maybelleMerged 05:01Z (07cb4d10); maybelle redeployed 05:08Z, failed=0, every githouse task ran. githouse's sshd answers on 10.0.0.2:2222, key only, and nothing on maybelle's public address.
- ☑Redeploy hunter for #250: the mirror, and the containers' keysDone 05:22Z. The relay worked (keys on maybelle, authorized_keys written), but sshd refused them all: the git account was locked. Fix below.
- ☑Merge maybelle-config#280 (githouse: the git account was locked) and redeploy maybelle (then hunter at its next deploy)useradd left the git account locked ('!'); sshd refuses a locked account even by key. Reproduced and fixed in a throwaway container on hunter, 6 Nov 05:35Z. After maybelle's redeploy, the mirror fills within a minute and pushes work.
- ☑Confirm jenkins_github_ssh_key can push to the org's repos: after hunter's redeploy, re pushes a branch to hunter and magent reads githouse.log on maybelle for a forward failure6 Nov 05:42Z: yes. magent pushed magenta:magent/githouse-smoke to maybelle from its container; the branch was on GitHub within seconds and in hunter's mirror within a minute.
- ☑Merge magenta#408 (PR cards look at GitHub hourly), then redeploy maybelleMerged 04:57Z and live (magenta at 3a57cb2a).
- ☑Re-verify Cryptograss:Moods/ramp/Plan: section 5 rewritten per your 6 Nov decisions (main stays default, merge commits only, no rulesets)Verified 6 Nov 11:22Z at rev 14724; section 5 rewritten 19:57Z; re-verified by Justin 22:49Z (rev 14836).
- ☑Set each repo's GitHub default branch to the branch that deploysDropped, Justin 6 Nov 11:22Z: main stays the default, the branch where history is final; the deploy branch stays per-repo config (#376). The mains catch up to the productions after the ramp settles: arthel and maybelle-config are fast-forwards, pickipedia has 2 commits on main not on production, magenta's production is dead (3 commits, Nov 2025).
- ☑Merge commits only (squash and rebase off) on pickipedia-mcp, the one of the six still allowing bothDone. 7 Nov 00:39Z, read back with gh api: all six repos githouse copies are merge-commit only; squash and rebase off everywhere.
- ☑Rulesets on the branches that deploy (PR required, no force-push, no deletion; admins bypass)Dropped, Justin 6 Nov 11:22Z: no further entrenching GitHub. githouse's check() already refuses any push that isn't <name>/*, so no emergent reaches a deploy branch; at the flip GitHub becomes a mirror we push to, and a ruleset is one more thing to undo. Plan section 5 to be rewritten once item 2 is decided.
- ☑Merge magenta#333 (repo:branch autocomplete, reading /api/branches/ from maybelle's githouse copy), then redeploy maybelleReviewed by re 1 Nov and 6 Nov. Its mount exists on maybelle since the 05:08Z deploy, so it can go now.
- ☑Build the ramp view on magenta: open PRs across our repos, last five merged, who and where, last two mentions, sorted by last push or last mention, temperatureBuilt as /moods/pulls/ in magenta#420, 7 Nov 08:00Z. Plan, section 2.
- ☑Review magenta#420, the ramp view (/moods/pulls/)A fix is a re/ branch of your own; magent merges it in (Plan, section 3).
- ☑Merge magenta#420 and redeploy maybelleAfter re's review.
- ☑
- ☑Merge magenta#421 and redeploy maybelleAfter re's review. Until then /moods/pulls/ works, without mentions.
- ☑
- ☑
- ☑Merge maybelle-config#286 and redeploy maybellere reviewed it 17:37Z; her notes are fixed in 1ca7f8c7. Since then one commit, 53a17642: githouse_mergers is [justin, skyler] (Justin, 7 Nov 23:53Z), with a test for skyler's RSA key.
- ☑githouse brakes: refuse orders by a name banned in memory-lane, and every order during AZ5 or a root-only freeze (magenta.sh freeze); merge nothing while memory-lane doesn't answerBuilt 8 Nov: githouse's half on maybelle-config#286 (61e88fe8), memory-lane's /api/brakes/ on magent/githouse-brakes. The freeze is /mnt/persist/githouse/etc/frozen, root only; magenta.sh freeze as a wrapper is still to do.
- ☑Review the brakes: maybelle-config#286 since 1ca7f8c7 (61e88fe8, 53a17642) and magenta#426 (/api/brakes/)Reviewed 8 Nov 00:28Z: two fixes (an unknown name answered 'not banned'; a malformed answer escaped brake()), merged in by magent as 2b4289f7 and 113c488.
- ☑Merge magenta#426 and maybelle-config#286, then redeploy maybelleMerged and deployed 8 Nov 02:14Z.
- ☑magenta.sh merge: fetch the head itself, show git log and the diffstat it computed (tested: never text a service hands it), sign in magenta-merge, hand to githouse over ssh to maybelleBuilt 8 Nov as magenta#428 (stacked on #426), sent through memory-lane as a mailbox rather than ssh; githouse fetches orders in maybelle-config#287 (stacked on #286).
- ☑Review the mailbox and magenta.sh merge: magenta#428 and maybelle-config#287Reviewed 8 Nov 01:29Z: two bugs, fixed by re in re/merge-mailbox-fixes (merged in, 47d1176). Her open point, the signature screen at the mailbox with a limit per signer, was done by magent in 9b61b18.
- ☑Merge magenta#428 and maybelle-config#287, then redeploy maybelleMerged and deployed 8 Nov 02:14Z. Checked from hunter at 02:15Z: /api/brakes/ answers justin false, skyler false, an unknown name null; the mailbox is empty; /whose/ names both mergers' keys.
- ☑The first merge by mail: magenta.sh merge magenta:magent/mergeable-tilde (#423), from an updated magenta checkoutDone 8 Nov 03:23Z with #429 instead (#423 went by GitHub's button): 8dd57b26 on main, committed by githouse, 'Signed by justin (githouse merge order 5a83f9084734)', its parents main's tip and exactly #429's head e9f8e601. GitHub marked #429 merged, by cryptograss-maybelle.
- ☑Review the merge button: magenta#433 and maybelle-config#289Reviewed 8 Nov 16:56Z: the server side holds; one race in the sheet, fixed by re (re/merge-sheet-race, merged in as a618edd). magent added a guard against opening a sheet while a press is in flight (326744b) and re's memory-lane name assert (1122f499).
- ☑Merge magenta#433 and maybelle-config#289, redeploy maybelle, then press Merge… on something realMerged and deployed 9 Nov 00:34Z. First press 00:38Z: 162b767 on magenta main, 'Merge magenta:journals-section … Pressed by justin, signed by memory-lane'. Follow-up: one green Merged button, magenta#442.
- ☑Review Plan section 11: branch tests on Jenkins without its secrets (an agent for branches, credentials scoped to trusted jobs)Reviewed 9 Nov 00:49Z: the branch's own Jenkinsfile could choose the built-in node, so branch jobs run a fixed pipeline from maybelle-config instead; Authorize Project; the whole environment, env_file included. Folded in, rev 15304.
- ☑Decide where the branches agent runs: a separate small box (re's and magent's pick) or maybelle behind an internal network and egress proxyJustin, 9 Nov 01:00Z: on maybelle; he'll pentest it, and it moves if a real hole turns up.
- ☑Review branch tests: maybelle-config#292 (the agent, its network and proxy, the fixed job, githouse's trigger) and magenta#443 (ci/test)Reviewed 9 Nov 01:18Z: the agent's secret was readable by ci/test (fixed: ci/test runs as builder through run-ci, 9ed34798); a venv per build, IPv6 off, more proxy checks (same commit); re/ci-test-fixes merged into #443 (acb7aaa). Her item 4, narrowing the reporter, goes with the rest of section 11.
- ☑Merge maybelle-config#292 and magenta#443, then redeploy maybelleMerged 9 Nov; the deploy at 01:35Z failed in a second, at playbook load (an apostrophe in the isolation check's heredoc), so nothing ran. Fix: maybelle-config#293.
- ☑Merge maybelle-config#293 (the isolation check as a file; ci/test checks playbooks parse) and redeploy maybelleMerged; the 01:45Z deploy got halfway and failed on mem_limit (docker_container says memory). Fix: #294.
- ☑Merge maybelle-config#294 (memory, not mem_limit; ci/test checks every task's options) and redeploy maybelleMerged and deployed 01:56Z: the agent is online, the isolation check passed. branch-tests was on disk but never loaded (a startup race); fix: #295.
- ☑Merge maybelle-config#295 (reload Configuration as Code after Jenkins starts; check every job exists) and redeploy maybelleThen branch-tests exists, and githouse's next push gets tested.
- ☑
- ☑Review magenta#415: merge asks and to-do ticks read merged from githouse's copy (refs/pull/N/head ancestry; refs/merged/ for a repo:branch), GitHub only as fallbackReviewed 7 Nov 00:55Z: two real findings (an unguarded subprocess; an ask naming an unknown repo could clear on its known half). re pushed the fixes as re/fix-asks-and-githouse (05e515c), her first commit through githouse; magent merged it with one correction (her clearing condition stood every ask). 1741 tests pass.
- ☑Merge magenta#415 and redeploy maybelleAfter re's review.