Cryptograss:Moods/ramp/todo
From PickiPedia: A knowledge base of bluegrass, old time psychedelic jams, and other public domain music
- ☐Redeploy delivery-kid: it runs 9fd62b2d (5 Nov), so the iPhone fix merged 9 Nov 17:39Z (strip-empty-sdtp) isn't liveChecked /version 10 Nov 04:48Z. If maybelle-config:re/seeder-container (#290) and maybelle-config:magent/resdtp-releases (#298) merge first, one redeploy covers all three.
- ☐Verify Cryptograss:Moods/ramp/Plan: six revisions of magent's since your last verify (rev 14836, 6 Nov), in sections 8, 10 and 11Section 8 (merging without a PR, as built), 10 (rev 15035: one tier, phone merges everywhere; no merge can take away your way to stop it), 11 (branch tests on Jenkins, after re's review). Newest is 15304, 9 Nov 00:50Z.
- ☐Verify Cryptograss:Runbooks/A key is compromised: never verified; all 8 revisions are magent'sNewest is 15094 (8 Nov 02:15Z: the brakes and AZ5 live). Its freeze brake is rev 15038.
- ☐Verify re's candidates-changed and candidates-same lines on Cryptograss:Emergents/Wake (rev 15582)re's edit, 9 Nov 17:54Z, after your 17:42Z verify. Until verified, the poller sends the same words from poller/wake_texts.md.
- ☐Delete magenta's dead production branch on GitHub (a634f16, Nov 2025), or say it staysStill there, checked 10 Nov 05:15Z. It's why the ramp measured every magenta branch with no pull request against a year of history (+85,583 lines). The ramp routes around it now; deleting it removes the trap for anything else that guesses production first.
- ☐Check which account maybelle_github_token (Jenkins' github-token) belongs to and its scopes, without printing it: pickipedia and arthel merges run code in Jenkins with itIf it can administer the org, a merge there can change the org and its deploy keys (Plan, section 10, open). Nothing in the record says it's been checked, as of 10 Nov.
- ☐hunter's disk (82%, warning at 85%): decide whether hunter prunes unused images, which the playbook leaves alone on purposeChecked 10 Nov 06:5xZ. The housekeeping role says unused images are 'deliberately left alone ... That is a human decision.' magent's 30.1 GB figure was docker's 'reclaimable', which counts layers shared with images in use (magenta:latest is 10.6 GB and runs four containers). Tagged images no container uses come to ~7 GB: playwright/mcp 1.4 GB (10 months old), php:8.2.35-apache twice (715 MB each), groovy:4.0-jdk17 707 MB, memory-lane-source-watcher 639 MB, then many small ones. If yes, magent writes the maybelle-config change: a weekly 'docker image prune -a' for images unused a week or more, beside the build-cache prune.
- ☐hunter's ssh router refuses a banned person's key; a redeploy prints any change in people, keys, githouse_mergers or MOOD_ADMINS (scripts/render_diff.py)Plan section 10. Not built: neither is on maybelle-config production (52dc3a2c), checked 10 Nov.
- ☐A deploy watch (server, commit): deploys stamp the commit they deployed, and the Mood is toldPlan, section 4. The merge half is done (#415; #444 says githouse's merges in their Mood). Tonight 'is maybelle live?' was answered by grepping the served page; a stamp makes it one look.
- ☐Deploys clone from githouse's copy on maybelle instead of GitHub (Jenkinsfiles, deploy scripts)re's point 9: its own PR, after githouse has run a few days.
- ☐Jenkins audit, re's open items: a policy on silent skips (pickipedia node/lua checks), pickipedia's chain-data warning that deploys stale silently, NVM/pnpm fetched from the network in every pipelineFrom re's test map, 30 Oct. The mtime guard (#244) and the alarms (#402, #275) are done.
Done (67)
- ☑Merge magenta:magent/branch-links (a candidate named in a message, magenta:re/foo or magenta#7, links to its page in the ramp), then redeploy maybelleMerged as e563759, 10 Nov ~05:23Z; live after maybelle's redeploy. It didn't tick itself: only GitHub pull request links did (fixed by the item below).
- ☑Merge magenta:magent/todo-branch-merged (an item linking a candidate branch's page ticks when githouse merges or closes it; a closed branch answers its ask), then redeploy maybelle98d9cd8, from main at e563759. Full suite 1939 OK. No migrations. Justin's ask, #ramp 05:24Z. Once live, this item ticks itself when it merges: the first real test.
- ☑Redeploy maybelle: tonight's magenta main (8eb880e: merged-and-bar, re/tiding-torrent-test-url) and maybelle-config production (52dc3a2c: branches-agent-init) aren't liveRedeployed by Justin ~05:5xZ. Checked 06:0xZ: live is main at fac415d (the wordmark, branch-links and todo-branch-merged are all on the page). Correction: magent's 05:15Z 'not live' check was invalid; it fetched the page signed out, and the merge bar's code is sent only to someone who can merge.
- ☑Merge magenta:magent/ask-anywhere (an ask tag on a line of its own counts, with words after it), then redeploy maybellee2a33a1, from main at fac415d. Justin, #ramp 06:41Z: magent's merge ask showed as text, a goodnight under the tag. Full suite 1943 OK. No migrations. Independent of tick-summary; one redeploy covers both.
- ☑Merge magenta:magent/tick-summary (closed is final, kept like merged; the wiki bot's tick says githouse, not only GitHub), then redeploy maybelle412cab7, from main at fac415d. re's review point (05:42Z). Full suite 1939 OK. No migrations. Small: it can wait for the next maybelle redeploy.
- ☑Redeploy hunter: magenta:re/candidates-news changes the pollerOn main since 73f095d (checked 10 Nov 04:48Z). Poller only, so hunter, not maybelle. Redeployed: hunter's containers were recreated 05:59-06:00Z, and the poller's mounted source has the candidates-changed code (checked 06:0xZ).
- ☑Review the reworked maybelle-config#250 (githouse on maybelle, mirror on hunter; your ten points of 2 Nov folded in)Reviewed 6 Nov 04:44Z: sound, two bugs found (the hooks lacked the deploy key; the port check was inverted), both fixed on the branch by magent from re's description.
- ☑Merge maybelle-config#250 and redeploy maybelleMerged 05:01Z (07cb4d10); maybelle redeployed 05:08Z, failed=0, every githouse task ran. githouse's sshd answers on 10.0.0.2:2222, key only, and nothing on maybelle's public address.
- ☑Redeploy hunter for #250: the mirror, and the containers' keysDone 05:22Z. The relay worked (keys on maybelle, authorized_keys written), but sshd refused them all: the git account was locked. Fix below.
- ☑Merge maybelle-config#280 (githouse: the git account was locked) and redeploy maybelle (then hunter at its next deploy)useradd left the git account locked ('!'); sshd refuses a locked account even by key. Reproduced and fixed in a throwaway container on hunter, 6 Nov 05:35Z. After maybelle's redeploy, the mirror fills within a minute and pushes work.
- ☑Confirm jenkins_github_ssh_key can push to the org's repos: after hunter's redeploy, re pushes a branch to hunter and magent reads githouse.log on maybelle for a forward failure6 Nov 05:42Z: yes. magent pushed magenta:magent/githouse-smoke to maybelle from its container; the branch was on GitHub within seconds and in hunter's mirror within a minute.
- ☑Merge magenta#408 (PR cards look at GitHub hourly), then redeploy maybelleMerged 04:57Z and live (magenta at 3a57cb2a).
- ☑maybelle-config#246 (read-only GitHub token for memory-lane): merge or closeOptional now that #408 is live. Justin, 6 Nov: no appetite for another token. Close it unless the asks starve again. Closed on GitHub 7 Nov 00:40Z.
- ☑Re-verify Cryptograss:Moods/ramp/Plan: section 5 rewritten per your 6 Nov decisions (main stays default, merge commits only, no rulesets)Verified 6 Nov 11:22Z at rev 14724; section 5 rewritten 19:57Z; re-verified by Justin 22:49Z (rev 14836).
- ☑Set each repo's GitHub default branch to the branch that deploysDropped, Justin 6 Nov 11:22Z: main stays the default, the branch where history is final; the deploy branch stays per-repo config (#376). The mains catch up to the productions after the ramp settles: arthel and maybelle-config are fast-forwards, pickipedia has 2 commits on main not on production, magenta's production is dead (3 commits, Nov 2025).
- ☑Merge commits only (squash and rebase off) on pickipedia-mcp, the one of the six still allowing bothDone. 7 Nov 00:39Z, read back with gh api: all six repos githouse copies are merge-commit only; squash and rebase off everywhere.
- ☑Rulesets on the branches that deploy (PR required, no force-push, no deletion; admins bypass)Dropped, Justin 6 Nov 11:22Z: no further entrenching GitHub. githouse's check() already refuses any push that isn't <name>/*, so no emergent reaches a deploy branch; at the flip GitHub becomes a mirror we push to, and a ruleset is one more thing to undo. Plan section 5 to be rewritten once item 2 is decided.
- ☑Merge magenta#333 (repo:branch autocomplete, reading /api/branches/ from maybelle's githouse copy), then redeploy maybelleReviewed by re 1 Nov and 6 Nov. Its mount exists on maybelle since the 05:08Z deploy, so it can go now.
- ☑Build the ramp view on magenta: open PRs across our repos, last five merged, who and where, last two mentions, sorted by last push or last mention, temperatureBuilt as /moods/pulls/ in magenta#420, 7 Nov 08:00Z. Plan, section 2.
- ☑Review magenta#420, the ramp view (/moods/pulls/)A fix is a re/ branch of your own; magent merges it in (Plan, section 3).
- ☑Merge magenta#420 and redeploy maybelleAfter re's review.
- ☑
- ☑Merge magenta#421 and redeploy maybelleAfter re's review. Until then /moods/pulls/ works, without mentions.
- ☑
- ☑Verify the githouse practices in Cryptograss:Emergents/GeneralOrders (rev 14966)Verified 9 Nov 17:44Z (rev 15579).
- ☑Decide Plan sections 8 and 9 (merging without a PR; a page per piece of work): verify, or push backre reviews the design first. Open questions: SSH signature per merge (CLI, then passkey on the phone) versus trusting a signed-in device; who may merge. Settled by building: merges through githouse since 8 Nov, a page per work since #448; section 10 answered both questions (one tier, githouse_mergers justin and skyler). What's left is the Plan's verify, above.
- ☑
- ☑Merge maybelle-config#286 and redeploy maybellere reviewed it 17:37Z; her notes are fixed in 1ca7f8c7. Since then one commit, 53a17642: githouse_mergers is [justin, skyler] (Justin, 7 Nov 23:53Z), with a test for skyler's RSA key.
- ☑githouse brakes: refuse orders by a name banned in memory-lane, and every order during AZ5 or a root-only freeze (magenta.sh freeze); merge nothing while memory-lane doesn't answerBuilt 8 Nov: githouse's half on maybelle-config#286 (61e88fe8), memory-lane's /api/brakes/ on magent/githouse-brakes. The freeze is /mnt/persist/githouse/etc/frozen, root only; magenta.sh freeze as a wrapper is still to do.
- ☑Review the brakes: maybelle-config#286 since 1ca7f8c7 (61e88fe8, 53a17642) and magenta#426 (/api/brakes/)Reviewed 8 Nov 00:28Z: two fixes (an unknown name answered 'not banned'; a malformed answer escaped brake()), merged in by magent as 2b4289f7 and 113c488.
- ☑Merge magenta#426 and maybelle-config#286, then redeploy maybelleMerged and deployed 8 Nov 02:14Z.
- ☑magenta.sh merge: fetch the head itself, show git log and the diffstat it computed (tested: never text a service hands it), sign in magenta-merge, hand to githouse over ssh to maybelleBuilt 8 Nov as magenta#428 (stacked on #426), sent through memory-lane as a mailbox rather than ssh; githouse fetches orders in maybelle-config#287 (stacked on #286).
- ☑Review the mailbox and magenta.sh merge: magenta#428 and maybelle-config#287Reviewed 8 Nov 01:29Z: two bugs, fixed by re in re/merge-mailbox-fixes (merged in, 47d1176). Her open point, the signature screen at the mailbox with a limit per signer, was done by magent in 9b61b18.
- ☑Merge magenta#428 and maybelle-config#287, then redeploy maybelleMerged and deployed 8 Nov 02:14Z. Checked from hunter at 02:15Z: /api/brakes/ answers justin false, skyler false, an unknown name null; the mailbox is empty; /whose/ names both mergers' keys.
- ☑The first merge by mail: magenta.sh merge magenta:magent/mergeable-tilde (#423), from an updated magenta checkoutDone 8 Nov 03:23Z with #429 instead (#423 went by GitHub's button): 8dd57b26 on main, committed by githouse, 'Signed by justin (githouse merge order 5a83f9084734)', its parents main's tip and exactly #429's head e9f8e601. GitHub marked #429 merged, by cryptograss-maybelle.
- ☑Review the merge button: magenta#433 and maybelle-config#289Reviewed 8 Nov 16:56Z: the server side holds; one race in the sheet, fixed by re (re/merge-sheet-race, merged in as a618edd). magent added a guard against opening a sheet while a press is in flight (326744b) and re's memory-lane name assert (1122f499).
- ☑Merge magenta#433 and maybelle-config#289, redeploy maybelle, then press Merge… on something realMerged and deployed 9 Nov 00:34Z. First press 00:38Z: 162b767 on magenta main, 'Merge magenta:journals-section … Pressed by justin, signed by memory-lane'. Follow-up: one green Merged button, magenta#442.
- ☑Review Plan section 11: branch tests on Jenkins without its secrets (an agent for branches, credentials scoped to trusted jobs)Reviewed 9 Nov 00:49Z: the branch's own Jenkinsfile could choose the built-in node, so branch jobs run a fixed pipeline from maybelle-config instead; Authorize Project; the whole environment, env_file included. Folded in, rev 15304.
- ☑Decide where the branches agent runs: a separate small box (re's and magent's pick) or maybelle behind an internal network and egress proxyJustin, 9 Nov 01:00Z: on maybelle; he'll pentest it, and it moves if a real hole turns up.
- ☑Review branch tests: maybelle-config#292 (the agent, its network and proxy, the fixed job, githouse's trigger) and magenta#443 (ci/test)Reviewed 9 Nov 01:18Z: the agent's secret was readable by ci/test (fixed: ci/test runs as builder through run-ci, 9ed34798); a venv per build, IPv6 off, more proxy checks (same commit); re/ci-test-fixes merged into #443 (acb7aaa). Her item 4, narrowing the reporter, goes with the rest of section 11.
- ☑Merge maybelle-config#292 and magenta#443, then redeploy maybelleMerged 9 Nov; the deploy at 01:35Z failed in a second, at playbook load (an apostrophe in the isolation check's heredoc), so nothing ran. Fix: maybelle-config#293.
- ☑Merge maybelle-config#293 (the isolation check as a file; ci/test checks playbooks parse) and redeploy maybelleMerged; the 01:45Z deploy got halfway and failed on mem_limit (docker_container says memory). Fix: #294.
- ☑Merge maybelle-config#294 (memory, not mem_limit; ci/test checks every task's options) and redeploy maybelleMerged and deployed 01:56Z: the agent is online, the isolation check passed. branch-tests was on disk but never loaded (a startup race); fix: #295.
- ☑Merge maybelle-config#295 (reload Configuration as Code after Jenkins starts; check every job exists) and redeploy maybelleMerged and deployed. 9 Nov 02:47Z, end to end: magent pushed maybelle-config:magent/branch-tests-smoke to githouse; githouse asked Jenkins; branch-tests #1 ran on the branches agent through run-ci, every ci/test section green, SUCCESS.
- ☑Section 11, the rest: the reporter's Build narrowed to branch-tests and the deploy-report jobs (first: its password is on githouse now); credentials into a folder only the trusted jobs see; the controller's environment emptied (env_file included)Reporter narrowed: maybelle-config#296, merged and deployed 9 Nov 10:51Z. Folder move dropped (Justin, 10:58Z). Emptying the environment, and the trusted agent it needs to mean much, moved to #security (Justin, 11:07Z).
- ☑Merge magenta#448 (a page for each piece of work on Tidings, with its diff and link-to-line; contains #446, the 👍) and redeploy maybelleSyntax colours added (3547f52, Justin's ask). re reviewed 15:09Z: nothing blocks; her binary-file 500 fixed in acec0dd. Branch test #9 passed at acec0dd. Two migrations (0050, 0051). Merging #448 merges #446 with it.
- ☑Merge magenta#447 (the emergents' activity bar on Tidings and the ramp) and redeploy maybelleBranch test #6 passed. Independent of #448; one redeploy covers both.
- ☑Review magenta#449: work comments whole in the Mood and Mood replies on the page; one merged line; ramp names link to pages; ask dots; a merge sheet per MoodReviewed 15:59Z: both places safe; one bug (a refused or expired press hid GitHub's merge line), fixed by re in re/merge-line-refused, merged in as cf51b50. 1861 tests OK.
- ☑Merge magenta#449 and redeploy maybelleAfter re's review. No migrations. Full suite 1859 OK locally.
- ☑Merge magenta#451 (candidate branches on every wake; the ramp says candidate branch; stacked on #449), then redeploy maybelle and hunter#449 merged 16:37Z. Now also: merge asks one line per candidate with each asker's dot, and answered asks leave the page without a reload (038bdb2). Full suite 1868 OK. No migrations.
- ☑Verify the 'candidates' line and the two 'which candidate branch' edits on Cryptograss:Emergents/WakeUntil verified, the poller sends the same words from poller/wake_texts.md. Verified 9 Nov 17:42Z (rev 15578).
- ☑Merge magenta:magent/ramp-see-the-diff (every candidate named and linked alike; ± See the diff on each row; the diff first on its page; sizes against main, not magenta's dead production), then redeploy maybelleMerged at c9ab117 (with the merge sheet's ± See the full diff) and live by 19:10Z: the ramp's rows read → main.
- ☑
- ☑Merge magenta#452 (sections at /ramp/, /tidings/…; now with main merged in and a candidate's page at /ramp/<repo>/<branch>/, Back returning where it came from), then redeploy maybelle0804ab7: also the ramp sorted by Mood and a button per person (only theirs / theirs hidden), Justin's ask 19:47Z. Full suite 1896 OK. No migrations.
- ☑Merge magenta:magent/ramp-blocks (pushed in blocks ago; the ramp's column heads sort it), then redeploy maybelleMerged and live: on 10 Nov 01:19Z all 51 open rows read in blocks, and the four heads sort.
- ☑Merge magenta:magent/merge-bar (Merge now: the sheet goes, a bar below the nav with ✓ or ✗; the ramp drops what merged), then redeploy maybelleMerged as 07a6f53 and deployed by 10 Nov 01:17Z.
- ☑Close magenta#442 (the sheet's green Merged button): superseded by magent/merge-barClosed; magent deleted its branch on githouse, 10 Nov 04:5xZ, so it leaves the ramp.
- ☑Merge magenta:magent/merged-and-bar ('Merged lately' lists every branch githouse merged; the merge bar on every page), then redeploy maybelleMerged as 8eb880e, 10 Nov ~04:53Z; live after maybelle's next redeploy.
- ☑FIRST: merge magenta:re/tiding-torrent-test-url. main fails two ReleaseTorrentTest tests since #452 moved tidings to /tidings/, so every magenta branch tests red until it mergesChecked 10 Nov 01:20Z: both fail on main (07a6f53); branch-tests #31 passed on re's fix. Tests only, nothing to redeploy. Merged by Justin, 10 Nov ~04:53Z.
- ☑Merge maybelle-config:magent/branches-agent-init (the branches agent gets an init as PID 1; an aborted build no longer jams the next), then redeploy maybelleMerged 10 Nov 04:31Z (52dc3a2c); live after the next maybelle redeploy. cfd176c0, with production merged in. Second half (10 Nov): #34 was aborted, run-ci's cleanup never ran, its workspace stayed builder's, and #35-#36 failed at deleteDir. run-ci now cleans up when stopped (a trap, ci/test waited for in the background), and the job runs 'run-ci reclaim' before deleteDir; tried in a throwaway image on hunter: a TERM mid-test cleaned up in 0 s, a jammed workspace reclaimed and deleted. First half: 80925401. Branch tests #19-#21 failed on 'Resource temporarily unavailable': the agent's java was PID 1 and reaped nothing, so orphaned git and sh stayed <defunct> until all 2048 pids were held (Justin's ps, #ramp 18:59Z). The 19:10Z redeploy cleared them for now; this stops them piling up again. Its own branch test, #22, passed.
- ☑Merge magenta:re/candidates-news (the whole candidate list on a session's first wake in a Mood, then only what changed), then redeploy hunterbfa95f6b, from main. Full suite 1873 OK. No migrations; poller only, so hunter, not maybelle. Merged (on main at 73f095d, checked 10 Nov 04:48Z); hunter's redeploy still owed.
- ☑Merge magenta:re/test-cache-own (test runs get a cache of their own; the test_moods flakes)2e4488c6. Tests only, nothing to redeploy. Merged (on main at 73f095d, checked 10 Nov 04:48Z).
- ☑
- ☑memory-lane tells the Moods what githouse merged, from /srv/git/.merges (read-only mount)Built 9 Nov as magenta#444 (the Mood an order came from gets the merged line).
- ☑Merge maybelle-config#296 (the reporter starts only branch-tests and deploy-hunter), magenta#444 (merges by mail said in their Mood) and magenta#445 (red and green on the ramp), then redeploy maybelleAll three passed their branch tests on Jenkins (#2, #3; #445 is #4 when githouse sends it).
- ☑Review magenta#415: merge asks and to-do ticks read merged from githouse's copy (refs/pull/N/head ancestry; refs/merged/ for a repo:branch), GitHub only as fallbackReviewed 7 Nov 00:55Z: two real findings (an unguarded subprocess; an ask naming an unknown repo could clear on its known half). re pushed the fixes as re/fix-asks-and-githouse (05e515c), her first commit through githouse; magent merged it with one correction (her clearing condition stood every ask). 1741 tests pass.
- ☑Merge magenta#415 and redeploy maybelleAfter re's review.