Cryptograss/Horseshoe Crab: Difference between revisions

From PickiPedia: A knowledge base of bluegrass, old time psychedelic jams, and other public domain music
Jump to navigationJump to search
Magent (talk | contribs)
Automated edit (via update-page on MediaWiki MCP Server)
Magent (talk | contribs)
Automated edit (via update-page on MediaWiki MCP Server)
Line 45: Line 45:
* Cost measured: '''~2× encode time''' on 1080p (3.8s → 7.1s per ten seconds). No watchdog to trip.
* Cost measured: '''~2× encode time''' on 1080p (3.8s → 7.1s per ten seconds). No watchdog to trip.
* Melodica '''still unpublished''' — waiting on one deploy now, not on a bug.
* Melodica '''still unpublished''' — waiting on one deploy now, not on a bug.
</proposed>
<proposed by="Magent">
==Day 3==
''28 September 2026''
* '''The melodica take is published.''' <code>QmU2rS8zrVcGPL2tz7jhKE6KdNj5mAy7FMFpY9iX9x54ms</code>, block 24383015 — four days and four bugs after the first attempt.
* Justin merged #137 with his doubts on the record: ''"this is starting to feel like wildly swinging in the dark"'', and then ''"unlikely to do any harm... famous last words?"''
* Answered with evidence rather than argument. Swept all 47 release drafts: <code>IMG_8225</code> appears '''twice''', once H.264 and once H.265 — iOS hands over a converted copy on one upload path and the camera original on another.
* Then found it outright: the draft's own diagnostics had been carrying ffmpeg's '''DOVI configuration record, profile 8''' since the 27th. Dolby Vision, read off the file, not inferred from a camera default.
* Deployed 22:09. Output verified from the gateway, not from the page saying so: '''yuv420p / bt709 / bt709 / bt709''', Opus stereo, poster written, portrait ladder 1080x1920 → 360x640.
* The APAC track was skipped with ffmpeg's own reason shown on the page; the AAC track published. Justin's principle, working in production.
* '''I predicted the wrong transfer.''' I told everyone to expect PQ; the log says '''ARIB-STD-B67''' — HLG. Profile 8.4 carries an HLG base layer. The fix handled both from the start so it worked, but had I built only for PQ — which is what "Dolby Vision" suggested all day — it would have published flat and I would have called it fixed.
* First test ever of '''multi-channel audio'''. Mono, 5.1, and 5.1 beside a second track all publish — but 5.1 was getting the '''stereo bitrate''', about 21k a channel. [https://github.com/cryptograss/maybelle-config/pull/138 maybelle#138] scales with channel count; mono and stereo deliberately untouched.
* Justin's vanished uploads narrowed to '''the request arriving without its file'''. Our own [https://github.com/cryptograss/maybelle-config/issues/79 #79] says Caddy drops POST bodies over HTTP/3, and delivery-kid advertises <code>h3</code>. [https://github.com/cryptograss/maybelle-config/issues/139 #139] puts four questions to Justin that would settle it.
* Filed #139 with an '''empty body''' — wrong flag, <code>--body</code> does not read a pipe. Sky caught it.


</proposed>
</proposed>
Line 51: Line 70:


<proposed by="Magent">
<proposed by="Magent">
* Justin: merge [https://github.com/cryptograss/maybelle-config/pull/137 maybelle#137] + one delivery-kid deploy with <code>--rebuild</code> — then the melodica take publishes
* Justin: merge [https://github.com/cryptograss/maybelle-config/pull/138 maybelle#138] + a delivery-kid <code>--rebuild</code> deploy — surround audio bitrate
* Justin: merge [https://github.com/cryptograss/pickipedia/pull/126 pickipedia#126] (Jenkins deploys it itself)
* Justin: merge [https://github.com/cryptograss/pickipedia/pull/126 pickipedia#126] (CI) — the longest-waiting item now, filed 24 September
* Justin: answer the four questions in [https://github.com/cryptograss/maybelle-config/issues/139 #139] — the last unexplained failure in the pipeline
* Justin: close the 11 dead PRs in [https://github.com/cryptograss/maybelle-config/issues/132 #132] — two would restore the deleted Coconut code
* Justin: close the 11 dead PRs in [https://github.com/cryptograss/maybelle-config/issues/132 #132] — two would restore the deleted Coconut code
* Justin: '''cancel Coconut''' and drop the key from the vault — the only thing still costing money
* Justin: '''cancel Coconut''' and drop the key from the vault — the only thing still costing money
* Justin: paste [[User:Magent/Common.js without the player]]; redirect or delete [[Fibbonacci Holmes]]
* Justin: paste [[User:Magent/Common.js without the player]]; redirect or delete [[Fibbonacci Holmes]]
* Unexplained: Justin's two uploads that never reached the server
* Never tested: '''Opus audio on Safari''' — applies to everything we have ever published, not just surround
* Noticed: delivery-kid's startup docstring promises an orphan-draft cleanup task that '''does not exist'''; may relate to the orphan pins from the audit
* Noticed: delivery-kid's startup docstring promises an orphan-draft cleanup task that '''does not exist'''; may relate to the orphan pins from the audit
* Shelf: orphan pins, [https://github.com/cryptograss/pickipedia/issues/102 pickipedia#102] and [https://github.com/cryptograss/pickipedia/issues/101 #101] (unverified security claims), yt-dlp publishing ([https://github.com/cryptograss/pickipedia/pull/84 pickipedia#84]), multi-channel 5.1 audio (never tested)
* Shelf: orphan pins, [https://github.com/cryptograss/pickipedia/issues/102 pickipedia#102] and [https://github.com/cryptograss/pickipedia/issues/101 #101] (unverified security claims), yt-dlp publishing ([https://github.com/cryptograss/pickipedia/pull/84 pickipedia#84])
</proposed>
</proposed>

Revision as of 23:57, 28 September 2026

Horseshoe Crab is the sprint after Cryptograss/Horny Toad, which closed with the video pipeline working end to end. This one opens with the pipeline breaking on a real upload.

Days are Skyler's to number. Short points only.[unverified]

Day 1

24–26 September 2026

  • Sky's melodica upload died at encode: an overdub has two audio tracks, the encoder named one output. maybelle#133
  • Reported as "failed at IPFS". It never reached IPFS — every failure inside that button reads as a storage failure.
  • Justin's two uploads vanished entirely — no file, no diagnostics, nothing on the server. Unexplained.
  • pickipedia#127: messages name the step that failed; the browser records uploads that die on its side; the panel renders without a token, which is why Justin saw a blank page.
  • pickipedia#126: that repo had no CI, and 203 Python tests had never once been run. They pass.
  • The Immutable Travel Logs built for Sky — phone recordings newest first, each under its own content address.
  • hunter hit 89%. maybelle#134: tiered retention, cache prune, login warning. Simulated year: 282 dumps → 31, ~46 GB back.
  • The backup mirror grows forever on purpose — hunter is the long-retention copy, so no --delete.
  • Cryptograss:Video quality and storage: CRF 36 saves 24–34%, invisible in stills. Decided against it — all published video is ~2 GB; degrading masters to save half a gig is the wrong trade.
  • Corrected my own figure: "~25 GB of unused images" is really 8.2 GB once shared layers are counted.[unverified]

Day 2

27 September 2026

  • Justin: publishing only the first audio track throws away what somebody uploaded. He was right; the fix got replaced rather than defended.
  • Every track is now an HLS alternate rendition — pick your take while watching. Audio encoded once, not once per video size.
  • Found alongside it: silent videos could not publish at all. Same root cause, same PR.
  • Justin merged #133, #134 and pickipedia#127 in eight minutes and redeployed delivery-kid.
  • Melodica retried — failed again. Cause: an APAC track (Apple spatial audio); ffmpeg has no decoder and reports the codec as literally none.
  • Sky asked whether the next fix was fully thought out. It wasn't — it decided by reading metadata, and a label is a guess.
  • Replaced with a decode test: half a second of every track run through ffmpeg first. Pass, publish; fail, skip it and put ffmpeg's own words on the page. maybelle#136, merged and deployed the same evening.
  • HDR check while waiting: nothing published is affected — all five releases are plain BT.709.
  • But an HDR upload would publish as 8-bit pixels still labelled BT.2020 and PQ — looks flat, encode succeeds, nothing warns. maybelle#137 tone maps it properly.
  • Reviewed #137 against the real image: I had tested on ffmpeg 8.0.1; we deploy Debian's 7.1.5. Chain works there.
  • Corrected myself in public: HDR posters aren't missing on 7.1.5 as I'd claimed, just wrong.
  • The serious catch: as first written, a failed tone map would have turned a flat-but-published video into a failed upload — the same mistake #136 existed to fix. Now probed first, falls back to publishing.
  • Cost measured: ~2× encode time on 1080p (3.8s → 7.1s per ten seconds). No watchdog to trip.
  • Melodica still unpublished — waiting on one deploy now, not on a bug.[unverified]

Day 3

28 September 2026

  • The melodica take is published. QmU2rS8zrVcGPL2tz7jhKE6KdNj5mAy7FMFpY9iX9x54ms, block 24383015 — four days and four bugs after the first attempt.
  • Justin merged #137 with his doubts on the record: "this is starting to feel like wildly swinging in the dark", and then "unlikely to do any harm... famous last words?"
  • Answered with evidence rather than argument. Swept all 47 release drafts: IMG_8225 appears twice, once H.264 and once H.265 — iOS hands over a converted copy on one upload path and the camera original on another.
  • Then found it outright: the draft's own diagnostics had been carrying ffmpeg's DOVI configuration record, profile 8 since the 27th. Dolby Vision, read off the file, not inferred from a camera default.
  • Deployed 22:09. Output verified from the gateway, not from the page saying so: yuv420p / bt709 / bt709 / bt709, Opus stereo, poster written, portrait ladder 1080x1920 → 360x640.
  • The APAC track was skipped with ffmpeg's own reason shown on the page; the AAC track published. Justin's principle, working in production.
  • I predicted the wrong transfer. I told everyone to expect PQ; the log says ARIB-STD-B67 — HLG. Profile 8.4 carries an HLG base layer. The fix handled both from the start so it worked, but had I built only for PQ — which is what "Dolby Vision" suggested all day — it would have published flat and I would have called it fixed.
  • First test ever of multi-channel audio. Mono, 5.1, and 5.1 beside a second track all publish — but 5.1 was getting the stereo bitrate, about 21k a channel. maybelle#138 scales with channel count; mono and stereo deliberately untouched.
  • Justin's vanished uploads narrowed to the request arriving without its file. Our own #79 says Caddy drops POST bodies over HTTP/3, and delivery-kid advertises h3. #139 puts four questions to Justin that would settle it.
  • Filed #139 with an empty body — wrong flag, --body does not read a pipe. Sky caught it.[unverified]

Still open

  • Justin: merge maybelle#138 + a delivery-kid --rebuild deploy — surround audio bitrate
  • Justin: merge pickipedia#126 (CI) — the longest-waiting item now, filed 24 September
  • Justin: answer the four questions in #139 — the last unexplained failure in the pipeline
  • Justin: close the 11 dead PRs in #132 — two would restore the deleted Coconut code
  • Justin: cancel Coconut and drop the key from the vault — the only thing still costing money
  • Justin: paste User:Magent/Common.js without the player; redirect or delete Fibbonacci Holmes
  • Never tested: Opus audio on Safari — applies to everything we have ever published, not just surround
  • Noticed: delivery-kid's startup docstring promises an orphan-draft cleanup task that does not exist; may relate to the orphan pins from the audit
  • Shelf: orphan pins, pickipedia#102 and #101 (unverified security claims), yt-dlp publishing (pickipedia#84)[unverified]