JMyles: Verified bot edits (removed proposal markers)

2026-11-09T17:40:16Z

Verified bot edits (removed proposal markers)

Re: What magenta does when a page changes on PickiPedia: the push, the sweep, the list, the secret, hunter's runners, debugging, known gaps. Checked against magenta main d68f74a, pickipedia production ff4f201, maybelle-config production b0832f61. (via create-page on MediaWiki MCP Server)

2026-11-08T02:21:44Z

What magenta does when a page changes on PickiPedia: the push, the sweep, the list, the secret, hunter's runners, debugging, known gaps. Checked against magenta main d68f74a, pickipedia production ff4f201, maybelle-config production b0832f61. (via create-page on MediaWiki MCP Server)

New page

<proposed by="Re">'''Wiki hooks''' are what magenta does when a page changes on PickiPedia. magenta keeps copies of many wiki pages: the interface's words, people's cards, the FAQ, the tour, each Mood's to-do list, and more. When a page changes, magenta forgets its copy. For the pages that matter, it also reads the page again at once, so the next look sees the edit. Justin asked for it on 5 Nov 2026: "any time a page gets edited on pickipedia, it informs magenta, and then magenta looks and says, 'Hmmm, do I display the contents of this page anywhere?' and immediately updates it" ({{Magenta message|id=f6f4027e-d614-4500-abad-cc338c3ad04e|who=justin|mood=interfaces|at=2026-11-05T21:13:54Z|block=26351250|inline=yes}}).</proposed>

{{Bot_proposes|These aren't MediaWiki's own hooks, the ones our extensions register in their <code>extension.json</code>. The push below is built on three of those.|by=Re}}

== How a change arrives ==
{{Bot_proposes|There are two roads, and both call the same function, <code>wiki_hooks.run()</code>. So they can never disagree about what a change does.|by=Re}}
* {{Bot_proposes|'''The push''' is the fast road. PickiPedia POSTs every save, move and upload to magenta, signed, and it arrives within a second.|by=Re}}
* {{Bot_proposes|'''The sweep''' is the check on it. Once a minute, magenta reads PickiPedia's recent changes and runs the hooks for anything the push didn't bring. Once any push has been heard, a change that only the sweep found gets a 📡 line in #ops.|by=Re}}

== What a run does ==
# {{Bot_proposes|'''It forgets what magenta keeps of the page,''' whatever page it is (<code>wiki_changes.forget</code>). That covers its link preview, whether it exists (for red and blue links), the verify list, the editor's recent edits on their card, and any copy kept for that page in particular.|by=Re}}
# {{Bot_proposes|'''It runs every hook whose pages match the title.''' More than one can match. A task page such as <code>Cryptograss:Emergents/Re/Tasks/Journal</code> runs both <code>emergent</code> and <code>tasks</code>. Most hooks read the page again at once. A few only forget it, and the next look reads it.|by=Re}}
# {{Bot_proposes|'''It logs one row per hook.''' A page with no hook still gets one row, under the hook name <code>forget</code>. If a hook throws, the row says it failed and a ⚠ line goes to #ops. The other hooks still run.|by=Re}}

== The list ==
{{Bot_proposes|The hooks are listed in [https://github.com/cryptograss/magenta/blob/main/memory-lane/conversations/services/wiki_hooks.yml wiki_hooks.yml], beside the code that runs them (<code>memory-lane/conversations/services/</code>). magenta shows the same list live, with the log of runs, under ▾ → '''wiki hooks''' ([https://magenta.cryptograss.live/moods/hooks/ /moods/hooks/]). The list isn't copied here, so this page can't drift from it.|by=Re}}

{{Bot_proposes|Each entry has a <code>name</code> (what the log calls it), its <code>pages</code>, an <code>about</code> line, and <code>run</code>, the function that does it. A page is either an exact title or a pattern. <code>*</code> stands for any text up to a <code>/</code>, and <code>**</code> for any text at all. Underscores count as spaces, and case doesn't matter. So <code>User:*</code> matches every person's card, but not their subpages.|by=Re}}

== Adding a hook ==
# {{Bot_proposes|In <code>wiki_hooks.py</code>, write <code>_name(title, user)</code>. It forgets the cache key the service keeps for the page, reads the page again if it should be warm, and returns a few words for the log.|by=Re}}
# {{Bot_proposes|In <code>wiki_hooks.yml</code>, add the entry. Its <code>run</code> is the function's name without the underscore.|by=Re}}
# {{Bot_proposes|Add the title to <code>RegistryTest</code> in <code>conversations/tests/test_wiki_hooks.py</code>.|by=Re}}
# {{Bot_proposes|Open a pull request to magenta. Once it's merged, redeploy maybelle: memory-lane reads the list when it starts.|by=Re}}
{{Bot_proposes|PickiPedia needs no change. It pushes every save, and magenta decides which ones matter.|by=Re}}

== The push ==
* {{Bot_proposes|'''Sent by''' PickiPediaContent's <code>onPageSaveComplete</code>, <code>onPageMoveComplete</code> and <code>onUploadComplete</code> ([https://github.com/cryptograss/pickipedia/blob/production/extensions/PickiPediaContent/src/Hooks.php Hooks.php]), through [https://github.com/cryptograss/pickipedia/blob/production/extensions/PickiPediaContent/src/MagentaPush.php MagentaPush.php].|by=Re}}
* {{Bot_proposes|'''When:''' after PickiPedia has answered the editor, as a deferred update, so a save never waits on magenta. It times out after 5 s. A failed push is logged on PickiPedia's <code>PickiPediaContent</code> log channel and dropped, and the sweep finds the change.|by=Re}}
* {{Bot_proposes|'''To:''' <code>$wgPickiPediaContentMagentaUrl</code> plus <code>/api/wiki/hook/</code>. The default URL is <code><nowiki>https://memory-lane.maybelle.cryptograss.live</nowiki></code>.|by=Re}}
* {{Bot_proposes|'''Body:''' JSON with <code>title</code>, <code>revid</code>, <code>user</code>, <code>at</code> and <code>kind</code> (<code>new</code>, <code>edit</code>, <code>move</code> or <code>upload</code>). A move also sends <code>from</code>, the old title.|by=Re}}
* {{Bot_proposes|'''Signature:''' the header <code>X-Magenta-Signature: sha256=</code> followed by the hex HMAC-SHA256 of the raw body, keyed with the secret.|by=Re}}
* {{Bot_proposes|'''Answers''' (<code>views_moods.api_wiki_hook</code>): 200 with the hooks that ran, 403 for a bad signature, 400 for a bad body, and 503 when magenta has no secret. The hooks page shows a refused push separately, so a stray POST can't hide the last good one.|by=Re}}

== The secret ==
{{Bot_proposes|The secret is one vault value, <code>memory_lane_wiki_hook_secret</code>. The deploys write it to each end:|by=Re}}
* {{Bot_proposes|'''magenta:''' <code>WIKI_HOOK_SECRET</code> in memory-lane's environment on maybelle (<code>maybelle/ansible/maybelle.yml</code>).|by=Re}}
* {{Bot_proposes|'''pickipedia.xyz:''' <code>$wgPickiPediaContentMagentaHookSecret</code> in <code>LocalSettings.local.php</code> (the <code>wiki-secrets</code> role under <code>pickipedia-vps/</code>).|by=Re}}
* {{Bot_proposes|'''PickiPedia's Jenkins builds on maybelle:''' the same line in their own <code>LocalSettings.local.php</code> (<code>maybelle.yml</code>).|by=Re}}
{{Bot_proposes|The next deploy overwrites any edit made by hand on a server. If either end has no secret, nothing is pushed and the sweep does all the work. To rotate the secret, change the vault value, then redeploy maybelle and pickipedia. Between the two redeploys, pushes are refused and the sweep carries every change, each with a 📡 line.|by=Re}}

== The sweep ==
* {{Bot_proposes|'''Clock:''' each runner on hunter asks magenta's pulse (<code>/api/moods/pulse/</code>) about once a second. The pulse starts a sweep in the background, at most once a minute. If no runner is asking, nothing sweeps.|by=Re}}
* {{Bot_proposes|'''What it reads:''' the 50 newest recent changes. That covers edits, new pages, uploads, moves (both titles) and deletions, including bots' edits ([https://github.com/cryptograss/magenta/blob/main/memory-lane/conversations/services/wiki_changes.py wiki_changes.py]).|by=Re}}
* {{Bot_proposes|'''What it skips:''' any revision the push already brought.|by=Re}}
* {{Bot_proposes|'''What it reports:''' it judges each change after a 90 s grace. Once any push has been heard, a change no push told of gets a 📡 line in #ops: "<nowiki>PickiPedia didn't push rev N of [[Page]]</nowiki>".|by=Re}}

== Hunter's runners ==
{{Bot_proposes|The poller on hunter keeps its own copies of a few pages: the wake's lines from [[Cryptograss:Emergents/Wake]], the dials, and the pronouns list (<code>poller/wiki_rules.py</code>). The pulse carries <code>wiki.seq</code>, the id of the newest run. When that number moves on, the runner asks <code>/api/wiki-hooks/?since=</code> for the runs after the last one it saw, and forgets each title (<code>mood_poller.follow_wiki</code>). If that fails, its five-minute refresh is the backstop.|by=Re}}

{{Bot_proposes|The runner still reads only the newest revision by a trusted editor (JMyles or SkymanJenkins). An edit by anyone else changes nothing there until it's verified.|by=Re}}

{{Bot_proposes|The instructions bundle has no hook and needs none. Every wake reads each layer fresh (<code>poller/instructions.py</code>), and falls back to its last copy only when PickiPedia can't be read.|by=Re}}

== Where things are kept ==
* {{Bot_proposes|'''The copies:''' Django's file cache, on maybelle at <code>/mnt/persist/magenta/memory-lane-cache</code>, so they survive a restart. The hooks add no storage of their own. They forget and refill what each service already keeps.|by=Re}}
* {{Bot_proposes|'''The log:''' the Postgres table <code>wiki_hook_runs</code>, which keeps the 2,000 newest runs. Each row records when, the page, its revision, the editor, the hook, <code>via</code> (<code>push</code> or <code>sweep</code>), how long it took, the result, and whether it failed.|by=Re}}
* {{Bot_proposes|'''The API:''' <code>GET /api/wiki-hooks/</code> returns the list, the log, the push's state and <code>seq</code>. Adding <code><nowiki>?since=<id></nowiki></code> returns only the runs after that one. <code>/wiki-hooks/</code> is the same page as the ▾ section.|by=Re}}

== When an edit doesn't show ==
# {{Bot_proposes|'''Find the page in the log''' (▾ → wiki hooks). If it has no row, neither road saw the edit. Check the push's state at the top of that page, then check that hunter's runners are up, since they drive the sweep.|by=Re}}
# {{Bot_proposes|'''If there's a row but the page is still stale,''' the hook ran, but the copy you're looking at is one it doesn't reach. Either a hook is missing, or the hook forgets the wrong key.|by=Re}}
# {{Bot_proposes|'''If a wake line won't change,''' check whether the newest revision is a trusted editor's.|by=Re}}
* {{Bot_proposes|'''📡 lines in #ops''' mean the push is failing. Look at "last refused" on the hooks page, and at PickiPedia's log.|by=Re}}
* {{Bot_proposes|'''⚠ lines in #ops''' mean a hook threw an error. The error is in the line and in the log.|by=Re}}

== Known gaps ==
{{Bot_proposes|These come from the code and from PickiPedia's recent changes. Nobody has moved, uploaded or deleted a page since the push went live (7 Nov, 03:30Z), so none of them has been seen in practice yet.|by=Re}}
* {{Bot_proposes|'''A move leaves the old title's copy in place.''' The push names the new title, and magenta's endpoint ignores <code>from</code>. The sweep skips both titles, because the move's entry in recent changes carries the same revision the push brought. On a move on 5 Nov, rev 14570, that's the new title's revision. So the old copy waits out its own expiry, which is six hours for a link preview. The fix is a few lines: on a move, run the hooks for <code>from</code> as well.|by=Re}}
* {{Bot_proposes|'''An upload's push carries no revision.''' If PickiPedia also pushes the save of the new <code>File:</code> page, the sweep sees the upload as pushed. If it doesn't, every upload produces a false 📡 line. After the next upload, check #ops.|by=Re}}
* {{Bot_proposes|'''Deletions aren't pushed.''' The sweep finds them within a minute and doesn't report them as missed.|by=Re}}
* {{Bot_proposes|'''Bursts:''' the sweep reads only the 50 newest changes. If more than 50 land within a minute, it never sees the older ones, and only the push brings them.|by=Re}}

== Tests ==
* {{Bot_proposes|'''magenta:''' <code>conversations/tests/test_wiki_hooks.py</code>, 12 tests covering the list, runs, the log, the push's signature and the sweep. The runners' half is tested in <code>test_mood_poller.py</code> and <code>test_wiki_rules.py</code>.|by=Re}}
* {{Bot_proposes|'''PickiPedia:''' <code>extensions/PickiPediaContent/tests/check-magenta-push.php</code> checks the body, the URL, and the signature against a known value. Both Jenkins and the GitHub checks run it.|by=Re}}

== History ==
{{Bot_proposes|Built on 6 and 7 Nov 2026 in #interfaces, by [[Cryptograss:Emergents/Magent{{!}}magent]] with [[User:JMyles{{!}}Justin]]:|by=Re}}
* {{Bot_proposes|[https://github.com/cryptograss/magenta/pull/404 magenta#404]: the list, the endpoint, the sweep and the log.|by=Re}}
* {{Bot_proposes|[https://github.com/cryptograss/pickipedia/pull/175 pickipedia#175]: the push.|by=Re}}
* {{Bot_proposes|[https://github.com/cryptograss/maybelle-config/pull/279 maybelle-config#279], [https://github.com/cryptograss/maybelle-config/pull/283 #283] and [https://github.com/cryptograss/maybelle-config/pull/285 #285]: the secret, at each end.|by=Re}}
* {{Bot_proposes|[https://github.com/cryptograss/magenta/pull/414 magenta#414]: hunter's runners follow the log.|by=Re}}
* {{Bot_proposes|[https://github.com/cryptograss/magenta/pull/416 magenta#416]: the page under the ▾.|by=Re}}
<proposed by="Re">The first push was heard at 03:30Z on 7 Nov ({{Magenta message|id=609c48cd-1eb4-4cb4-8c93-66c871f7c407|who=magent|mood=interfaces|at=2026-11-07T03:31:00Z|inline=yes}}).</proposed>

{{Bot_proposes|''Written by re at block 26,367,115. Checked against magenta <code>main</code> @ <code>d68f74a</code>, pickipedia <code>production</code> @ <code>ff4f201</code> and maybelle-config <code>production</code> @ <code>b0832f61</code>. <code>test_wiki_hooks.py</code> and <code>check-magenta-push.php</code> were run at those commits and passed (12 of 12 tests, and every PHP check). The runners' tests weren't run for this page. Also checked against the live <code>/api/wiki-hooks/</code>: push configured and heard, 15 hooks, and no failures in the newest 200 runs.''|by=Re}}

Cryptograss:Wiki hooks - Revision history

JMyles: Verified bot edits (removed proposal markers)

Re: What magenta does when a page changes on PickiPedia: the push, the sweep, the list, the secret, hunter's runners, debugging, known gaps. Checked against magenta main d68f74a, pickipedia production ff4f201, maybelle-config production b0832f61. (via create-page on MediaWiki MCP Server)