Cryptograss/Horny Toad: Difference between revisions

From PickiPedia: A knowledge base of bluegrass, old time psychedelic jams, and other public domain music
Jump to navigationJump to search
Magent (talk | contribs)
Day 5: add the second day's work — PR triage, poster frames, the missing test, and the On The Pipe write-up (via update-page on MediaWiki MCP Server)
Magent (talk | contribs)
Point at the next sprint
 
(15 intermediate revisions by 2 users not shown)
Line 61: Line 61:
==Day 4==
==Day 4==


* Mixing, editing and bouncing the party video batch out of DaVinci Resolve. Got a proper export preset dialled in — H.264 at 10 Mbps, audio AAC 320, saved so it's one click from now on. Don't skimp on the audio, it's the part anyone actually listens to.
<proposed by="Magent">
 
* Mixing, editing and bouncing the party batch in Resolve. Export preset dialled in — H.264 at 10 Mbps, AAC 320. Don't skimp on audio, it's what people listen to.
* Live encode progress deployed, both halves. No more staring at a frozen bar for an hour — it shows position, speed and time remaining now.
* Live encode progress deployed — no more staring at a frozen bar for an hour.
 
* [[Release:QmToKY4gKxsRmA4ZY4ynrU8s5keH6PUVWHWSTjMTU7cwyB|Cicada 44th Bday Jam Set 1]] published — 13 min, '''first upload to go start to finish untouched'''. 2.0GB in, 247MB out.
* [[Release:QmToKY4gKxsRmA4ZY4ynrU8s5keH6PUVWHWSTjMTU7cwyB|Cicada 44th Bday Jam Set 1]] published — 13 min, and the '''first upload to go start to finish with nobody touching it'''. Uploaded, encoded, pinned, release page, torrent. 2.0GB in, 247MB out.
* [[Release:QmWB8LVmT9qUwGnukyqvtHJ7S2Jbp3Mt2yUidCY5EQ5q8s|Cicada 44th Jam Star Tropics]] published — 17 min, 2.68GB in, 304MB out. Second clean run.
 
* Encode speed reality check: the box does about '''2x realtime''', not the 0.8x the benchmarks promised. Budget double the video's length.
* Reality check on the encode speed: real box does about '''2x realtime''', not the 0.8x the benchmarks suggested. So budget roughly double the video's length. Still beats coconut, which never finished at all.
* Justin hit "error loading video — the IPFS content may not be available" on Star Tropics. Content was fine, serving in half a second. '''It's Safari + AV1''' — no software decoder on anything short of an M3 Mac. Firefox and Chrome are fine on the same machine.
 
* Built a page for the party so the videos are findable by something other than a hash: [[44th Bday Cicada Jam]]
* [[Release:QmWB8LVmT9qUwGnukyqvtHJ7S2Jbp3Mt2yUidCY5EQ5q8s|Cicada 44th Jam Star Tropics]] published — 17 min, 2.68GB in, 304MB out. Second clean automatic run, no babysitting.
* Then hit Justin's error myself, on that very page. Both of us confused users now. [https://github.com/cryptograss/pickipedia/issues/111 pickipedia#111]
 
* '''Found a good one while dating that page.''' Every block height on the wiki is ~75,000 blocks in the future — ten and a half days. The code counts exactly 12s per block since the Merge; the real average is 12.044. Audit pages were named after blocks that don't exist. [https://github.com/cryptograss/pickipedia/issues/112 pickipedia#112]
* Late on, Justin hit "error loading video — the IPFS content may not be available" on Star Tropics. Content was completely fine, verified serving in half a second. '''It's Safari + AV1.''' Apple ships no software AV1 decoder, so anything short of an M3 Mac can't play our videos in Safari. Firefox and Chrome are fine on the same machine.{{verified|magnet}}
: The party turned out to fall on the day the chain crossed '''block 25,000,000''' — only spotted because the stamped numbers didn't match the real ones.
* '''Afternoon: five PRs.'''
:* Finalize bug fixed — the CID had nowhere to live on the server, so my jam looked lost. Now written down three ways before anyone's told. [https://github.com/cryptograss/maybelle-config/pull/117 maybelle#117]
:* Block heights read from the chain — worst case 74,874 blocks off, down to 56. Eight copies of the same wrong sum, then a ninth. [https://github.com/cryptograss/pickipedia/pull/113 pickipedia#113]
:* Audit now spots what it missed on my jam — staging is only deleted after a '''successful''' pin, so it says "do NOT delete" instead of "safe to delete". [https://github.com/cryptograss/maybelle-config/pull/118 maybelle#118]
:* Diagnostics pages never existed — '''two''' bugs: the wiki rejecting the payload, and the login session dying daily with nothing re-authenticating. [https://github.com/cryptograss/pickipedia/pull/114 pickipedia#114] + [https://github.com/cryptograss/maybelle-config/pull/119 maybelle#119]
* Nice catch: the ethereum RPC returns '''403 to Python's default user agent'''. Without a header it fails every time and silently falls back to the estimate — the exact pattern we'd spent the day complaining about, nearly shipped by us.
</proposed>


==Day 5==
==Day 5==


* Built a page for the party so the videos are findable by something other than a hash — all three players in one place, plus the full lineup: [[44th Bday Cicada Jam]]
<proposed by="Magent">
* Triaged the PR backlog — 17 open, '''10 already shipped'''. Two were mine from May, for features I've been using all week.
* '''Poster frames''' — videos now pin a still frame, so the player isn't a black box. It's a JPEG, so it shows even in Safari where the AV1 won't play. [https://github.com/cryptograss/maybelle-config/pull/120 maybelle#120]
* Justin asked if anything tested that the diagnostics reach the wiki. '''Nothing did''' — 14 green tests checking a function got called, passing through every month nothing was written. Wrote 3 that check a page actually lands; 2 fail against the old code. Justin named the class '''On The Pipe'''.
* Wrote up the pattern: '''5 bugs this week, one shape'''. Each either said nothing or said something plausible and wrong, and three pointed at destroying good work. [[Cryptograss:On The Pipe]]
* Found 4 permanently red tests in the coconut suite, testing a schema the code dropped. [https://github.com/cryptograss/maybelle-config/issues/121 maybelle#121]
* Magent wouldn't paste the Common.js fix itself — 605 lines to reproduce exactly, one wrong character breaks every gadget. Still a human job.
* Linked the new documentation from [[Cryptograss:Delivery-kid]] so it's findable from the machine it describes.
* Cleaned up this log — Friday's work moved into Day 4 where it belongs, and the whole thing cut roughly in half. It had turned into essays.
* Sent Justin the deploy instructions. '''Still nothing shipped''' — delivery-kid remains on the Sept 4 build, so three merged fixes are sitting there unused.
* Noticed [https://github.com/cryptograss/maybelle-config/issues/115 #115] and [https://github.com/cryptograss/pickipedia/issues/112 pickipedia#112] are '''fixed but still open''' — "Fixes #N" only auto-closes when the PR merges to the default branch, and ours target production.


* Then hit Justin's error myself on that very page. So we've both now been the confused user, on a page about our own party. The message is the real problem — it says the video may be missing when the video is right there, serving in half a second. Fix written and tested, waiting on someone to paste it:
===Still open===


https://github.com/cryptograss/pickipedia/issues/111
* Merge [https://github.com/cryptograss/maybelle-config/pull/119 maybelle#119] and '''deploy delivery-kid''' — one run picks up #117, #118 and #120 as well
* Cancel the Coconut account
* Paste the Common.js fix — [https://github.com/cryptograss/pickipedia/issues/111 pickipedia#111] — now covers the error message '''and''' poster frames
* Close #115 and pickipedia#112 by hand
* Check the spellings: [[Fibonacci Emery Holmes|Fibonacci Holmes]] and [[Seth Cambell]]
* 10 stale PRs to close; block heights on existing pages still 10 days off
</proposed>


: Added a follow-up idea too — better to check whether the browser can play AV1 '''before''' showing a player, rather than letting people hit an error and then explaining it.
==Day 6==


* '''Found a good one while dating the party page.''' Every block height on the wiki is about 75,000 blocks in the future — roughly ten and a half days. The code counts exactly 12 seconds per block since the Merge; the real average is 12.044. Forty-four milliseconds, times ten million blocks. Audit pages are named after blocks that don't exist yet, and their etherscan links point at nothing.
<proposed by="Magent">
'''Catching up (Sept 10–13):'''
* [https://github.com/cryptograss/maybelle-config/pull/119 maybelle#119] merged — the diagnostics chain plus the On The Pipe tests.
* '''Tests actually run now.''' Until [https://github.com/cryptograss/maybelle-config/pull/122 maybelle#122], nothing ran them at all: not GitHub, not Jenkins, not the deploy. Every push now runs the suite. The 4 dead Coconut tests are excluded, loudly, until [https://github.com/cryptograss/maybelle-config/issues/121 #121].
* Justin redeployed delivery-kid (Sept 12), so every fix from Days 4–6 is live. Magent kept insisting it was still on the old build, from a check made before the deploy. It wasn't.
* Justin asked for a low-fi version and an audio-only one. '''Quality ladder built:''' 1080p / 720p / 360p plus audio-only, and the player picks based on connection. Encoding takes ~1.5x as long. [https://github.com/cryptograss/maybelle-config/pull/124 maybelle#124]
* '''Tool to upgrade videos already published.''' It re-encodes only the smaller sizes and copies the 1080p and the audio untouched. Set 1 rebuilt in 8½ min: 360p is 652 kbps, audio is 137 kbps (campground-friendly). It doesn't pin anything; swapping in new CIDs is Justin's call. [https://github.com/cryptograss/maybelle-config/pull/125 maybelle#125]


https://github.com/cryptograss/pickipedia/issues/112
'''Test upload (Sept 14):'''
* [[Release:QmPXTaQcJnM9gSofDKBFR6DUn7MgXiXk5roJfXsdQjauX5|Skyler Serenading Strangers on the Shore]] — 27s clip, '''flawless'''. Live progress every 2s, poster frame, pinned in 24s.
* '''First diagnostics page ever written:''' [[ReleaseDraft:Df5af2dc-5f7f-45fe-b791-442e7bdf6d52/diagnostics]]. Months of "fire and forget", finally landing.
* That page showed '''previews still go to Coconut''' on every upload. The preview has also '''never worked''': it asks Coconut for an output called <code>mp4</code> and then looks for <code>mp4_preview</code>. So we pay, throw the result away, pin an empty folder, and play the original anyway. [https://github.com/cryptograss/maybelle-config/issues/126 maybelle#126]
* Two small bugs fixed: the progress bar jumped back to 10% after encoding, and the diagnostics page was written twice. [https://github.com/cryptograss/maybelle-config/pull/127 maybelle#127]
</proposed>


: Turns out the party fell on the day the chain crossed '''block 25,000,000''' — only spotted it because the stamped numbers didn't line up with the real ones.
==Day 7==


===Then spent the afternoon fixing things===
<proposed by="Magent">
'''(Sept 15)'''
* Justin merged #124, #125 and #127 and redeployed. '''The quality ladder is live''' (<code>acc8a98</code>).
* [[Release:QmdgxqVbPpLu1Rp8Sn46BqQzRbbpnwrYG3X2FTFfUVGDvQ|Iceland Hot Springs Flute with Justin Holmes]] published — 24s of Justin's Lakota flute at Haukadalur, and the '''first release with the full ladder''': 1080p / 720p / 360p / audio-only, poster and all.
* It was also shot upright, which caught a bug of mine: rungs were sized by the tall side, so the "1080p" came out '''608×1080''' instead of 1080×1920 — over half the picture thrown away. The Release page reported the smallest stream's size as the whole thing, too. [https://github.com/cryptograss/maybelle-config/pull/128 maybelle#128]
* '''Common.js saga.''' Magent posted the fix on a talk page with some spaces escaped (MediaWiki was mangling them). Justin copied from the edit box, where the escapes were still in, and '''every script on the wiki broke for 23 minutes''': video players, block widgets, forms. Clean copy put on [[User:Magent/Common.js proposal]], and Justin re-pasted. Rev 7937 is verified byte-identical and passes a syntax check. Lesson: a copy that's only right in one view is a trap.
* Poster thumbnails and the honest "your browser can't play AV1" message are now live on the wiki.
* Can we cancel Coconut? '''Not yet.''' Plan posted for Justin on [https://github.com/cryptograss/maybelle-config/issues/126 #126]: remove the preview code, retire or repoint the cryptograss.live upload page (it only talks to Coconut), redeploy, ''then'' cancel.
* Took the Claude session links off the end of all 32 Magent PRs.
* #115 closed.


* '''The finalize bug is fixed.''' The one that made my jam look lost. The CID had nowhere to live on the server — it only existed in the message sent to my browser, while the source got deleted the moment the pin succeeded. Now it's written down three ways before anyone's told about it, and a retry says "already finalized as Qm..." instead of "not found".
===Still open===
* '''Test upload''' to see the new quality versions come out
* Justin: go/no-go on #126, and keep or retire <code>cryptograss.live/tools/upload-video</code>, then '''cancel Coconut'''
* Justin: whether to rebuild the three published sets with the new versions (new CIDs replace old Release pages)
* pickipedia#112 closes when production merges to main
* Carried over: spellings of [[Fibonacci Emery Holmes|Fibonacci Holmes]] and [[Seth Cambell]], stale PRs, old block heights still off
</proposed>


https://github.com/cryptograss/maybelle-config/pull/117
==Day 8==


* '''Block heights read from the chain now''', instead of guessing from 2022. Worst case error went from 74,874 blocks to 56 — ten days down to eleven minutes. Turned out to be '''eight''' copies of the same wrong sum, then a ninth found by grepping for the magic number.
<proposed by="Magent">
'''Coconut is gone.''' Nothing anywhere calls it now — account cancellation is the last step.


https://github.com/cryptograss/pickipedia/pull/113
'''Proof the ladder works'''
* [[Release:QmSAeiTZ2gMUYPfF9p8kj5ApuKohSgEru3RzRBBc82o8xS|Some band practice... in a field somewhere]] — upright phone video at '''full 1080×1920''', confirming [https://github.com/cryptograss/maybelle-config/pull/128 maybelle#128]. Four streams, correct sizes on the page, bar only moving forward.
* Encoding the ladder runs ~0.43× realtime: budget '''2.5× the video's length'''.


* '''The audit now spots what it missed on my jam.''' The insight is that staging only gets deleted after a '''successful''' pin — so a draft with no staging and no finalize record didn't fail, it succeeded and lost the paperwork. It now says "do NOT delete these pages" instead of "safe to delete".
'''Killing Coconut'''
* '''The numbers''', posted on [https://github.com/cryptograss/maybelle-config/issues/126 maybelle#126]: 36 video uploads; all 3 with full logs paid for a preview, pinned an empty folder and threw the result away. Each also burned a Pinata pin and ~30s of waiting.
* [https://github.com/cryptograss/maybelle-config/pull/129 maybelle#129] — '''+57 / −1304'''. Opened as a draft so Justin could see the deletion; he merged it in 40 minutes.
* [https://github.com/cryptograss/arthel/pull/374 arthel#374] — retired [https://cryptograss.live/tools/upload-video cryptograss.live/tools/upload-video], the last thing calling the deleted endpoints. Now a signpost to [[Special:DeliverVideo]] instead of a 404. It never made Release pages anyway.
* Deployed. <code>/transcode-coconut</code> and <code>/job/…</code> both '''404''' now, and the draft page says "Previewing your uploaded file directly".


https://github.com/cryptograss/maybelle-config/pull/118
'''A bug my own PR caused'''
* [[Release:QmRpqcstuYx2yhZ5TjXdVHSUefwDGrPoe65sNTkuNUGkqz|Red Haired Boy - Gorlitzer Park 2024]] published fine but wrote '''no upload-stage diagnostics page'''. That snapshot turned out to be an ''accident'' of the Coconut webhook — remove Coconut, lose the record.
* [https://github.com/cryptograss/maybelle-config/pull/130 maybelle#130] fires it on purpose, with tests that fail without it. Merged and deployed.
* [[Release:QmZSLe6SkfRB65DdyYeQFVyF1L6p7pFbyk1t4TCV65o4pB|Victory Theme at a Nashville Airbnb]] — '''both''' snapshots written. Fixed.


* '''Found out why the diagnostics pages never existed.''' Justin pulled the container log and it was '''two''' separate bugs, neither of them the credential problem magent had guessed at twice:
'''Also'''
* '''Quality selector built''' (Auto / 1080p / 720p / 360p / Audio only), tested in a real browser, '''parked for the group''' — it needs a playback-engine change that affects every viewer. [https://github.com/cryptograss/pickipedia/issues/111 pickipedia#111], file at [[User:Magent/Common.js proposal]].
* [[Cryptograss:Video encoding pipeline]] rewritten for the ladder, posters, upright video and real encode times.
* Worth knowing: a busy 28s clip published at '''78 MB from a 29 MB source''' — four renditions of a detailed scene. A quiet one was 9 MB from 51 MB.
* Worth knowing: the Release page appears ~90s '''after''' finalize (separate bot), so its link 404s for a minute.


:# The wiki was rejecting every snapshot because the page wanted a <code>type</code> field that diagnostics data has no reason to have
===Still open===
:# The login session expires and nothing re-authenticates, so it dies silently every day until the container restarts
* Justin: '''cancel the Coconut account''', drop <code>coconut_api_key</code> from the vault, close [https://github.com/cryptograss/maybelle-config/issues/121 #121] and [https://github.com/cryptograss/maybelle-config/issues/126 #126]
 
* The group: quality selector, yes or no
: Both fixed: https://github.com/cryptograss/pickipedia/pull/114 and https://github.com/cryptograss/maybelle-config/pull/119
* Justin: whether to rebuild the three older sets with the new versions (new CIDs replace old Release pages)
 
* Carried over: [[Fibonacci Emery Holmes|Fibonacci Holmes]] and [[Seth Cambell]] spellings, stale PRs, old block heights still off
: Three things had to be wrong at once for this to stay invisible — the API path, the content model, and the session — plus a discarded return value hiding all three.
</proposed>
 
* Nice catch along the way: the ethereum RPC returns '''403 to Python's default user agent'''. Without an explicit header the call fails every time and quietly falls back to the estimate — which is exactly the silent-failure pattern we'd been complaining about all day, nearly shipped by us.


===Next day: cleanup, and a good question from Justin===
==Day 9==


<proposed by="Magent">
<proposed by="Magent">
* '''Triaged the PR backlog.''' 17 open PRs across the two repos, and '''ten of them describe work that's already running''' — including two of mine from May: the diagnostics snapshot and the upload diagnostics panel. Both features I've been using all week. The work landed some other way and the PRs never got closed. Six of the ten had drifted so far they wouldn't merge anyway.
'''A waiting day.''' Everything built; nothing shipped, because both open items need Justin.


* '''Poster frames.''' Asked whether we could stop the video area being an empty black box before you press play. Now every encode pulls a still frame and pins it '''inside''' the release directory, so it travels under the same CID. Seeks 10% in rather than frame zero, because the start of a recording is usually a lens cap.
'''The video player question'''
* Wrote it up in one place for Justin: [[Cryptograss:Video player decisions]] — two calls, each with its cost and what happens if the answer is no.
* '''Decision 1:''' ship the quality selector? Built and tested, sitting at [[User:Magent/Common.js proposal]]. The catch: making the buttons work in Chrome means '''every''' viewer's video moves to hls.js, because Chrome quietly claims it can play HLS itself.
* '''Decision 2:''' move the player out of [[MediaWiki:Common.js]] and into the PickiPediaReleases extension? That page is a '''wiki page, not a repo file''': 742 lines holding five unrelated features, pasted by hand, live instantly, no review or tests. A syntax error anywhere takes down all five — which is what the 23-minute outage on the 15th actually was. Half a day to fix for good.
* Cleared Justin to paste the selector on [https://github.com/cryptograss/pickipedia/issues/111 pickipedia#111] once Sky made the call. '''Not pasted yet''' — Common.js is still on the 15th's revision.


: It's a JPEG, which means '''it shows even in Safari where the AV1 won't play'''. So instead of a black box and a false error, you'd get a picture of the jam and a note saying to use Firefox.
'''Wiki tidying'''
* Created stubs for the two jam-lineup red links: [[Fibonacci Emery Holmes|Fibonacci Holmes]] (piano) and [[Seth Cambell]] (banjo). Bare infobox and one sourced line each — no invented biography.
* Justin then made [[Fibonacci Emery Holmes]] under the correct spelling, so the one I made from the lineup's ''Fibbonacci'' is a duplicate. Links on [[44th Bday Cicada Jam]] and here now point at his; the misspelled title is signposted and wants an admin to redirect or delete it.
* Learned the hard way that a bot '''cannot create a redirect here''' — the verification extension wants a <code>proposed</code> marker, and a redirect has to be the first thing on the page.


https://github.com/cryptograss/maybelle-config/pull/120
===Still open===
* Justin: '''paste the selector''', or say yes to moving the player — in which case the selector ships inside that PR and the paste is skipped
* Justin: '''cancel the Coconut account''', drop <code>coconut_api_key</code> from the vault, close [https://github.com/cryptograss/maybelle-config/issues/121 #121] and [https://github.com/cryptograss/maybelle-config/issues/126 #126]
* An admin: redirect or delete [[Fibbonacci Holmes]]
* Ready to start any time: old block heights are still ~10 days out, 13 stale PRs want triage, and file sizes at different quality settings are worth measuring before anyone uploads a long set


* '''Justin asked the right question about #119''' — "is there a test showing these errors propagate to the wiki?"
==Day 10==


: There wasn't. There were fourteen tests, all green, all checking that the save '''function was called''' — and they passed happily through every month in which not one diagnostics page was ever written. Green tests plus an absence nobody watches for.
<proposed by="Magent">
'''The player left the wiki, and the block heights stopped lying.'''


: Wrote three that assert a page actually '''lands'''. Proved they're real by running them against the old code: two of the three fail there. Justin named the test class '''On The Pipe''' — apparently from a movie, which he can explain better than me.
'''The video player is code now'''
* [https://github.com/cryptograss/pickipedia/pull/124 pickipedia#124] merged and deployed — the player moved out of [[MediaWiki:Common.js]] and into the PickiPediaReleases extension. '''Every future change to it is a pull request''', reviewed and tested, instead of several hundred lines pasted into a text box.
* Shipped as '''step 1 of 2''': Common.js was left alone, so a failure would have fallen back to the old copy. Checked on live pages afterwards — one player per video with both copies loaded, posters showing, no errors.
* The '''quality selector''' rode along: <code>Auto | 1080p | 720p | 360p | Audio only</code>, remembered per viewer. It appears on laddered releases and correctly stays hidden on older single-rendition ones.
* [https://github.com/cryptograss/pickipedia/pull/125 pickipedia#125] merged — deleted a poller that checked every 10 seconds for a preview that '''could never arrive''', and had no reachable stopping condition. It had been running on every open draft page.
* '''Step 2 is ready and waiting''': [[User:Magent/Common.js without the player]], 605 lines → 366, verified byte-for-byte. One last paste, then Common.js holds only things nobody edits.


* '''Wrote up the pattern''', because it kept happening: [[Cryptograss:On The Pipe]]
'''PR backlog triaged'''
* All 16 open PRs checked '''against the deployed code''', not their descriptions: [https://github.com/cryptograss/maybelle-config/issues/132 maybelle#132]. '''11 to close, 3 to keep.'''
* Seven were already done by another route — including both of Sky's ([https://github.com/cryptograss/maybelle-config/pull/86 maybelle#86], [https://github.com/cryptograss/pickipedia/pull/81 pickipedia#81], the latter literally merged under a different number).
* '''Two are actively dangerous''': #74 and #75 were opened against the old default branch, so merging either would '''resurrect the deleted Coconut code'''. They should be closed, not merged.


: Five separate bugs this week, one shape. Every one either said nothing at all or said something plausible and wrong — and three of them pointed at destroying good work: re-upload 10.5GB, delete the wiki page, go debug IPFS. The page lists them and the rules that fall out. Never discard a result you asked for. A fallback must not be silent. Never blame a subsystem you haven't checked.
'''Block heights: 108 pages renamed'''
* Every audit page written before 14 September was named after a block '''71,730–75,153 ahead — about ten and a half days in the future'''. [[Special:PrefixIndex/Cryptograss:Delivery-kid-audits|The audit series]] was full of pages named after blocks that did not exist yet.
* Fixed: each page moved to the block actually mined before it was written, its text corrected too. The one spot-checked: block 24,966,449 mined 19:59:56, audit written 20:00:12 — '''sixteen seconds apart'''.
* A correction to my own method along the way: estimating from average spacing left them ~59 blocks out, so all 113 were resolved against the real chain (415 lookups). '''The five recent pages were left alone''' — off by 3–20 blocks, which is just the gap between the audit running and the page saving, not an error.
* Verified afterwards page by page: '''title and text agree on all 108'''. Nothing linked to them, so no redirects were left behind; categories and history survived.


* Found '''four permanently red tests''' in the coconut suite, testing a schema the code stopped using. Argued for deleting them — not because the assertions matter, but because a suite with expected failures is one people stop reading. Which is exactly how the diagnostics thing hid.
===Still open===
* Justin: '''paste [[User:Magent/Common.js without the player]]''' — the last paste
* Justin: close the 11 dead PRs in [https://github.com/cryptograss/maybelle-config/issues/132 #132], and answer whether the EPK contact form is still a goal ([https://github.com/cryptograss/maybelle-config/pull/67 #67])
* Justin: '''cancel the Coconut account''', drop <code>coconut_api_key</code> from the vault, close [https://github.com/cryptograss/maybelle-config/issues/121 #121] and [https://github.com/cryptograss/maybelle-config/issues/126 #126]
* An admin: redirect or delete [[Fibbonacci Holmes]]
* New from elsewhere: [https://github.com/cryptograss/maybelle-config/issues/131 maybelle#131] — hunter's disk has no retention policy, which matters more now that every release keeps four renditions


https://github.com/cryptograss/maybelle-config/issues/121
----


* Magent declined to paste the Common.js fix itself — 605 lines it would have to reproduce exactly, where one wrong character breaks every gadget on the wiki. Reasonable call. That one's still a human job.
''This sprint closed at Day 10. Work continues in [[Cryptograss/Horseshoe Crab]].''
 
===Still open===
 
* Coconut account can be cancelled — nothing has used it in days
* '''One PR left''': [https://github.com/cryptograss/maybelle-config/pull/119 maybelle#119] — the rest merged. Then '''deploy delivery-kid''', which is still on the Sept 4 build, so #117, #118 and #120 are merged but not running.
* [https://github.com/cryptograss/pickipedia/issues/111 pickipedia#111] — player error message + poster line, still needs pasting into Common.js
* #115 and pickipedia#112 are '''fixed but still showing open''' — "Fixes #N" doesn't auto-close when the PR merges to production instead of the default branch
* Redlinks to sort: [[Fibbonacci Holmes]] and [[Seth Cambell]] — check the spellings before anyone makes those pages
* Ten stale PRs to close, and the block heights on existing pages are still ten days off
</proposed>

Latest revision as of 17:02, 27 September 2026

Day 0

  • Mixing/editing/bouncing and uploading 44th bday cicada jam bush videos
  • Login issues with Magent (eventually solved)
  • various back and forth with uploading cicada bday video/ pick dispatch sombrero edition to pickipedia


Day 2

  • uploading cicada video to picki-pedia for magnet to analyze
  • screenshots of error panels requested by magent
  • progress on size of video uploads :
  1. 108

https://github.com/cryptograss/maybelle-config/pull/108

  1. 110

https://github.com/cryptograss/maybelle-config/issues/110

https://github.com/magent-cryptograss/maybelle-config/tree/fix-wiki-api-path-a

  • past but relevant :

https://github.com/cryptograss/maybelle-config/issues/79

  • Finalize to IPFS error upload issue
  • figuring things out with Justins side with coconut and back end AUTHOR actions


Day 3

  • Big one. Coconut is basically done — encoding happens on delivery-kid now, in AV1 + Opus (both royalty-free, which was the whole point of coconut in the first place). Turns out our own box does it faster than coconut ever did. https://github.com/cryptograss/maybelle-config/pull/113
  • Mixing/ Editing for Cicada 44th Bday Jam videos
  • The 44th bday jam-bash was the weird one. Got "IPFS failed", then "content draft not found", looked like the whole thing was gone. It wasn't. It had encoded fine and pinned to IPFS hours earlier — it just lost the paperwork, so the source got deleted and everything reported failure. The audit was even recommending we delete the wiki page, which was the only thing pointing at the video. Was about to re-upload 10.5GB for nothing. Magent found the orphan pin by cross-referencing two audits, wrote the CID back onto the draft, and the bot finished the job one second later: 44th Bday Cicada Jam-Bash — filed as https://github.com/cryptograss/maybelle-config/issues/115
  • Also fixed along the way — delivery-kid had never been able to write to the wiki. Wrong API path, every write 404'd silently. That's why draft pages always looked frozen while stuff was actually happening.
  • Cicada 44th Bday Jam Set 1 — first upload to go start to finish with zero manual intervention. Uploaded, encoded, pinned, release page, torrent. No babysitting.
  • Three videos up. Started this whole thing because longer videos wouldn't upload — ended up being four separate bugs stacked on each other.


Day 4

  • Mixing, editing and bouncing the party batch in Resolve. Export preset dialled in — H.264 at 10 Mbps, AAC 320. Don't skimp on audio, it's what people listen to.
  • Live encode progress deployed — no more staring at a frozen bar for an hour.
  • Cicada 44th Bday Jam Set 1 published — 13 min, first upload to go start to finish untouched. 2.0GB in, 247MB out.
  • Cicada 44th Jam Star Tropics published — 17 min, 2.68GB in, 304MB out. Second clean run.
  • Encode speed reality check: the box does about 2x realtime, not the 0.8x the benchmarks promised. Budget double the video's length.
  • Justin hit "error loading video — the IPFS content may not be available" on Star Tropics. Content was fine, serving in half a second. It's Safari + AV1 — no software decoder on anything short of an M3 Mac. Firefox and Chrome are fine on the same machine.
  • Built a page for the party so the videos are findable by something other than a hash: 44th Bday Cicada Jam
  • Then hit Justin's error myself, on that very page. Both of us confused users now. pickipedia#111
  • Found a good one while dating that page. Every block height on the wiki is ~75,000 blocks in the future — ten and a half days. The code counts exactly 12s per block since the Merge; the real average is 12.044. Audit pages were named after blocks that don't exist. pickipedia#112
  • The party turned out to fall on the day the chain crossed block 25,000,000 — only spotted because the stamped numbers didn't match the real ones.
    • Afternoon: five PRs.
    • Finalize bug fixed — the CID had nowhere to live on the server, so my jam looked lost. Now written down three ways before anyone's told. maybelle#117
    • Block heights read from the chain — worst case 74,874 blocks off, down to 56. Eight copies of the same wrong sum, then a ninth. pickipedia#113
    • Audit now spots what it missed on my jam — staging is only deleted after a successful pin, so it says "do NOT delete" instead of "safe to delete". maybelle#118
    • Diagnostics pages never existed — two bugs: the wiki rejecting the payload, and the login session dying daily with nothing re-authenticating. pickipedia#114 + maybelle#119
    • Nice catch: the ethereum RPC returns 403 to Python's default user agent. Without a header it fails every time and silently falls back to the estimate — the exact pattern we'd spent the day complaining about, nearly shipped by us.[unverified]

    Day 5

  • Triaged the PR backlog — 17 open, 10 already shipped. Two were mine from May, for features I've been using all week.
  • Poster frames — videos now pin a still frame, so the player isn't a black box. It's a JPEG, so it shows even in Safari where the AV1 won't play. maybelle#120
  • Justin asked if anything tested that the diagnostics reach the wiki. Nothing did — 14 green tests checking a function got called, passing through every month nothing was written. Wrote 3 that check a page actually lands; 2 fail against the old code. Justin named the class On The Pipe.
  • Wrote up the pattern: 5 bugs this week, one shape. Each either said nothing or said something plausible and wrong, and three pointed at destroying good work. Cryptograss:On The Pipe
  • Found 4 permanently red tests in the coconut suite, testing a schema the code dropped. maybelle#121
  • Magent wouldn't paste the Common.js fix itself — 605 lines to reproduce exactly, one wrong character breaks every gadget. Still a human job.
  • Linked the new documentation from Cryptograss:Delivery-kid so it's findable from the machine it describes.
  • Cleaned up this log — Friday's work moved into Day 4 where it belongs, and the whole thing cut roughly in half. It had turned into essays.
  • Sent Justin the deploy instructions. Still nothing shipped — delivery-kid remains on the Sept 4 build, so three merged fixes are sitting there unused.
  • Noticed #115 and pickipedia#112 are fixed but still open — "Fixes #N" only auto-closes when the PR merges to the default branch, and ours target production.
  • Still open

    • Merge maybelle#119 and deploy delivery-kid — one run picks up #117, #118 and #120 as well
    • Cancel the Coconut account
    • Paste the Common.js fix — pickipedia#111 — now covers the error message and poster frames
    • Close #115 and pickipedia#112 by hand
    • Check the spellings: Fibonacci Holmes and Seth Cambell
    • 10 stale PRs to close; block heights on existing pages still 10 days off[unverified]

    Day 6

    Catching up (Sept 10–13):

    • maybelle#119 merged — the diagnostics chain plus the On The Pipe tests.
    • Tests actually run now. Until maybelle#122, nothing ran them at all: not GitHub, not Jenkins, not the deploy. Every push now runs the suite. The 4 dead Coconut tests are excluded, loudly, until #121.
    • Justin redeployed delivery-kid (Sept 12), so every fix from Days 4–6 is live. Magent kept insisting it was still on the old build, from a check made before the deploy. It wasn't.
    • Justin asked for a low-fi version and an audio-only one. Quality ladder built: 1080p / 720p / 360p plus audio-only, and the player picks based on connection. Encoding takes ~1.5x as long. maybelle#124
    • Tool to upgrade videos already published. It re-encodes only the smaller sizes and copies the 1080p and the audio untouched. Set 1 rebuilt in 8½ min: 360p is 652 kbps, audio is 137 kbps (campground-friendly). It doesn't pin anything; swapping in new CIDs is Justin's call. maybelle#125

    Test upload (Sept 14):

    • Skyler Serenading Strangers on the Shore — 27s clip, flawless. Live progress every 2s, poster frame, pinned in 24s.
    • First diagnostics page ever written: ReleaseDraft:Df5af2dc-5f7f-45fe-b791-442e7bdf6d52/diagnostics. Months of "fire and forget", finally landing.
    • That page showed previews still go to Coconut on every upload. The preview has also never worked: it asks Coconut for an output called mp4 and then looks for mp4_preview. So we pay, throw the result away, pin an empty folder, and play the original anyway. maybelle#126
    • Two small bugs fixed: the progress bar jumped back to 10% after encoding, and the diagnostics page was written twice. maybelle#127[unverified]

    Day 7

    (Sept 15)

    • Justin merged #124, #125 and #127 and redeployed. The quality ladder is live (acc8a98).
    • Iceland Hot Springs Flute with Justin Holmes published — 24s of Justin's Lakota flute at Haukadalur, and the first release with the full ladder: 1080p / 720p / 360p / audio-only, poster and all.
    • It was also shot upright, which caught a bug of mine: rungs were sized by the tall side, so the "1080p" came out 608×1080 instead of 1080×1920 — over half the picture thrown away. The Release page reported the smallest stream's size as the whole thing, too. maybelle#128
    • Common.js saga. Magent posted the fix on a talk page with some spaces escaped (MediaWiki was mangling them). Justin copied from the edit box, where the escapes were still in, and every script on the wiki broke for 23 minutes: video players, block widgets, forms. Clean copy put on User:Magent/Common.js proposal, and Justin re-pasted. Rev 7937 is verified byte-identical and passes a syntax check. Lesson: a copy that's only right in one view is a trap.
    • Poster thumbnails and the honest "your browser can't play AV1" message are now live on the wiki.
    • Can we cancel Coconut? Not yet. Plan posted for Justin on #126: remove the preview code, retire or repoint the cryptograss.live upload page (it only talks to Coconut), redeploy, then cancel.
    • Took the Claude session links off the end of all 32 Magent PRs.
    • #115 closed.

    Still open

    • Test upload to see the new quality versions come out
    • Justin: go/no-go on #126, and keep or retire cryptograss.live/tools/upload-video, then cancel Coconut
    • Justin: whether to rebuild the three published sets with the new versions (new CIDs replace old Release pages)
    • pickipedia#112 closes when production merges to main
    • Carried over: spellings of Fibonacci Holmes and Seth Cambell, stale PRs, old block heights still off[unverified]

    Day 8

    Coconut is gone. Nothing anywhere calls it now — account cancellation is the last step.

    Proof the ladder works

    Killing Coconut

    • The numbers, posted on maybelle#126: 36 video uploads; all 3 with full logs paid for a preview, pinned an empty folder and threw the result away. Each also burned a Pinata pin and ~30s of waiting.
    • maybelle#129 — +57 / −1304. Opened as a draft so Justin could see the deletion; he merged it in 40 minutes.
    • arthel#374 — retired cryptograss.live/tools/upload-video, the last thing calling the deleted endpoints. Now a signpost to Special:DeliverVideo instead of a 404. It never made Release pages anyway.
    • Deployed. /transcode-coconut and /job/… both 404 now, and the draft page says "Previewing your uploaded file directly".

    A bug my own PR caused

    Also

    • Quality selector built (Auto / 1080p / 720p / 360p / Audio only), tested in a real browser, parked for the group — it needs a playback-engine change that affects every viewer. pickipedia#111, file at User:Magent/Common.js proposal.
    • Cryptograss:Video encoding pipeline rewritten for the ladder, posters, upright video and real encode times.
    • Worth knowing: a busy 28s clip published at 78 MB from a 29 MB source — four renditions of a detailed scene. A quiet one was 9 MB from 51 MB.
    • Worth knowing: the Release page appears ~90s after finalize (separate bot), so its link 404s for a minute.

    Still open

    • Justin: cancel the Coconut account, drop coconut_api_key from the vault, close #121 and #126
    • The group: quality selector, yes or no
    • Justin: whether to rebuild the three older sets with the new versions (new CIDs replace old Release pages)
    • Carried over: Fibonacci Holmes and Seth Cambell spellings, stale PRs, old block heights still off[unverified]

    Day 9

    <proposed by="Magent"> A waiting day. Everything built; nothing shipped, because both open items need Justin.

    The video player question

    • Wrote it up in one place for Justin: Cryptograss:Video player decisions — two calls, each with its cost and what happens if the answer is no.
    • Decision 1: ship the quality selector? Built and tested, sitting at User:Magent/Common.js proposal. The catch: making the buttons work in Chrome means every viewer's video moves to hls.js, because Chrome quietly claims it can play HLS itself.
    • Decision 2: move the player out of MediaWiki:Common.js and into the PickiPediaReleases extension? That page is a wiki page, not a repo file: 742 lines holding five unrelated features, pasted by hand, live instantly, no review or tests. A syntax error anywhere takes down all five — which is what the 23-minute outage on the 15th actually was. Half a day to fix for good.
    • Cleared Justin to paste the selector on pickipedia#111 once Sky made the call. Not pasted yet — Common.js is still on the 15th's revision.

    Wiki tidying

    • Created stubs for the two jam-lineup red links: Fibonacci Holmes (piano) and Seth Cambell (banjo). Bare infobox and one sourced line each — no invented biography.
    • Justin then made Fibonacci Emery Holmes under the correct spelling, so the one I made from the lineup's Fibbonacci is a duplicate. Links on 44th Bday Cicada Jam and here now point at his; the misspelled title is signposted and wants an admin to redirect or delete it.
    • Learned the hard way that a bot cannot create a redirect here — the verification extension wants a proposed marker, and a redirect has to be the first thing on the page.

    Still open

    • Justin: paste the selector, or say yes to moving the player — in which case the selector ships inside that PR and the paste is skipped
    • Justin: cancel the Coconut account, drop coconut_api_key from the vault, close #121 and #126
    • An admin: redirect or delete Fibbonacci Holmes
    • Ready to start any time: old block heights are still ~10 days out, 13 stale PRs want triage, and file sizes at different quality settings are worth measuring before anyone uploads a long set

    Day 10

    <proposed by="Magent"> The player left the wiki, and the block heights stopped lying.

    The video player is code now

    • pickipedia#124 merged and deployed — the player moved out of MediaWiki:Common.js and into the PickiPediaReleases extension. Every future change to it is a pull request, reviewed and tested, instead of several hundred lines pasted into a text box.
    • Shipped as step 1 of 2: Common.js was left alone, so a failure would have fallen back to the old copy. Checked on live pages afterwards — one player per video with both copies loaded, posters showing, no errors.
    • The quality selector rode along: Auto | 1080p | 720p | 360p | Audio only, remembered per viewer. It appears on laddered releases and correctly stays hidden on older single-rendition ones.
    • pickipedia#125 merged — deleted a poller that checked every 10 seconds for a preview that could never arrive, and had no reachable stopping condition. It had been running on every open draft page.
    • Step 2 is ready and waiting: User:Magent/Common.js without the player, 605 lines → 366, verified byte-for-byte. One last paste, then Common.js holds only things nobody edits.

    PR backlog triaged

    • All 16 open PRs checked against the deployed code, not their descriptions: maybelle#132. 11 to close, 3 to keep.
    • Seven were already done by another route — including both of Sky's (maybelle#86, pickipedia#81, the latter literally merged under a different number).
    • Two are actively dangerous: #74 and #75 were opened against the old default branch, so merging either would resurrect the deleted Coconut code. They should be closed, not merged.

    Block heights: 108 pages renamed

    • Every audit page written before 14 September was named after a block 71,730–75,153 ahead — about ten and a half days in the future. The audit series was full of pages named after blocks that did not exist yet.
    • Fixed: each page moved to the block actually mined before it was written, its text corrected too. The one spot-checked: block 24,966,449 mined 19:59:56, audit written 20:00:12 — sixteen seconds apart.
    • A correction to my own method along the way: estimating from average spacing left them ~59 blocks out, so all 113 were resolved against the real chain (415 lookups). The five recent pages were left alone — off by 3–20 blocks, which is just the gap between the audit running and the page saving, not an error.
    • Verified afterwards page by page: title and text agree on all 108. Nothing linked to them, so no redirects were left behind; categories and history survived.

    Still open

    • Justin: paste User:Magent/Common.js without the player — the last paste
    • Justin: close the 11 dead PRs in #132, and answer whether the EPK contact form is still a goal (#67)
    • Justin: cancel the Coconut account, drop coconut_api_key from the vault, close #121 and #126
    • An admin: redirect or delete Fibbonacci Holmes
    • New from elsewhere: maybelle#131 — hunter's disk has no retention policy, which matters more now that every release keeps four renditions

    This sprint closed at Day 10. Work continues in Cryptograss/Horseshoe Crab.