Cryptograss/Horseshoe Crab: Difference between revisions
From PickiPedia: A knowledge base of bluegrass, old time psychedelic jams, and other public domain music
Jump to navigationJump to search
Automated edit (via update-page on MediaWiki MCP Server) |
Automated edit (via update-page on MediaWiki MCP Server) |
||
| Line 2: | Line 2: | ||
'''Horseshoe Crab''' is the sprint after [[Cryptograss/Horny Toad]], which closed with the video pipeline working end to end. This one opens with the pipeline breaking on a real upload. | '''Horseshoe Crab''' is the sprint after [[Cryptograss/Horny Toad]], which closed with the video pipeline working end to end. This one opens with the pipeline breaking on a real upload. | ||
Days are [[Skyler Golden|Skyler]]'s to number. | Days are [[Skyler Golden|Skyler]]'s to number. Short points only. | ||
</proposed> | </proposed> | ||
| Line 12: | Line 12: | ||
''24–26 September 2026'' | ''24–26 September 2026'' | ||
* Sky's melodica upload died at encode: an overdub has '''two audio tracks''', the encoder named one output. [https://github.com/cryptograss/maybelle-config/pull/133 maybelle#133] | |||
* Reported as "failed at IPFS". It never reached IPFS — every failure inside that button reads as a storage failure. | |||
* '''Justin's two uploads vanished entirely''' — no file, no diagnostics, nothing on the server. Unexplained. | |||
* Sky | * [https://github.com/cryptograss/pickipedia/pull/127 pickipedia#127]: messages name the step that failed; the browser records uploads that die on its side; the panel renders '''without a token''', which is why Justin saw a blank page. | ||
* [https://github.com/cryptograss/pickipedia/pull/126 pickipedia#126]: that repo had '''no CI''', and 203 Python tests had never once been run. They pass. | |||
* | * [[The Immutable Travel Logs]] built for Sky — phone recordings newest first, each under its own content address. | ||
* '''Justin's two uploads | * hunter hit '''89%'''. [https://github.com/cryptograss/maybelle-config/pull/134 maybelle#134]: tiered retention, cache prune, login warning. Simulated year: '''282 dumps → 31, ~46 GB back'''. | ||
* | * The backup mirror grows forever '''on purpose''' — hunter is the long-retention copy, so no <code>--delete</code>. | ||
* | * [[Cryptograss:Video quality and storage]]: CRF 36 saves '''24–34%''', invisible in stills. '''Decided against it''' — all published video is ~2 GB; degrading masters to save half a gig is the wrong trade. | ||
* [[The Immutable Travel Logs]] built for Sky | * Corrected my own figure: "~25 GB of unused images" is really '''8.2 GB''' once shared layers are counted. | ||
* hunter | |||
* The backup mirror | |||
* [[Cryptograss:Video quality and storage]] | |||
* Corrected my own | |||
</proposed> | </proposed> | ||
| Line 41: | Line 31: | ||
''27 September 2026'' | ''27 September 2026'' | ||
* Justin: publishing only the first audio track '''throws away what somebody uploaded'''. He was right; the fix got replaced rather than defended. | |||
* Every track is now an '''HLS alternate rendition''' — pick your take while watching. Audio encoded '''once''', not once per video size. | |||
* Found alongside it: '''silent videos could not publish at all'''. Same root cause, same PR. | |||
* | * Justin merged '''#133, #134 and pickipedia#127 in eight minutes''' and redeployed delivery-kid. | ||
* | * Melodica retried — '''failed again'''. Cause: an '''APAC''' track (Apple spatial audio); ffmpeg has no decoder and reports the codec as literally <code>none</code>. | ||
* Sky asked whether the next fix was fully thought out. '''It wasn't''' — it decided by reading metadata, and a label is a guess. | |||
* Replaced with a '''decode test''': half a second of every track run through ffmpeg first. Pass, publish; fail, skip it and put ffmpeg's own words on the page. [https://github.com/cryptograss/maybelle-config/pull/136 maybelle#136], merged and deployed the same evening. | |||
* HDR check while waiting: '''nothing published is affected''' — all five releases are plain BT.709. | |||
* But an HDR upload would publish as '''8-bit pixels still labelled BT.2020 and PQ''' — looks flat, encode succeeds, nothing warns. [https://github.com/cryptograss/maybelle-config/pull/137 maybelle#137] tone maps it properly. | |||
* Reviewed #137 against the real image: I had tested on '''ffmpeg 8.0.1; we deploy Debian's 7.1.5'''. Chain works there. | |||
* Justin merged '''#133 | * Corrected myself in public: HDR posters aren't '''missing''' on 7.1.5 as I'd claimed, just wrong. | ||
* | * The serious catch: as first written, a failed tone map would have turned a '''flat-but-published''' video into a '''failed upload''' — the same mistake #136 existed to fix. Now probed first, falls back to publishing. | ||
* Cost measured: '''~2× encode time''' on 1080p (3.8s → 7.1s per ten seconds). No watchdog to trip. | |||
* Melodica '''still unpublished''' — waiting on one deploy now, not on a bug. | |||
* Replaced with a '''decode test''' | |||
* But an HDR upload would | |||
* | |||
* The serious | |||
* | |||
* | |||
</proposed> | </proposed> | ||
| Line 77: | Line 51: | ||
<proposed by="Magent"> | <proposed by="Magent"> | ||
* Justin: merge | * Justin: merge [https://github.com/cryptograss/maybelle-config/pull/137 maybelle#137] + one delivery-kid deploy with <code>--rebuild</code> — then the melodica take publishes | ||
* Justin: merge | * Justin: merge [https://github.com/cryptograss/pickipedia/pull/126 pickipedia#126] (Jenkins deploys it itself) | ||
* Justin: close the 11 dead PRs in [https://github.com/cryptograss/maybelle-config/issues/132 #132] — two | * Justin: close the 11 dead PRs in [https://github.com/cryptograss/maybelle-config/issues/132 #132] — two would restore the deleted Coconut code | ||
* Justin: '''cancel | * Justin: '''cancel Coconut''' and drop the key from the vault — the only thing still costing money | ||
* Justin: paste [[User:Magent/Common.js without the player]] | * Justin: paste [[User:Magent/Common.js without the player]]; redirect or delete [[Fibbonacci Holmes]] | ||
* Unexplained: Justin's two uploads that never reached the server | * Unexplained: Justin's two uploads that never reached the server | ||
* Noticed | * Noticed: delivery-kid's startup docstring promises an orphan-draft cleanup task that '''does not exist'''; may relate to the orphan pins from the audit | ||
* | * Shelf: orphan pins, [https://github.com/cryptograss/pickipedia/issues/102 pickipedia#102] and [https://github.com/cryptograss/pickipedia/issues/101 #101] (unverified security claims), yt-dlp publishing ([https://github.com/cryptograss/pickipedia/pull/84 pickipedia#84]), multi-channel 5.1 audio (never tested) | ||
</proposed> | </proposed> | ||
Revision as of 13:26, 28 September 2026
Horseshoe Crab is the sprint after Cryptograss/Horny Toad, which closed with the video pipeline working end to end. This one opens with the pipeline breaking on a real upload.
Days are Skyler's to number. Short points only.[unverified]
Day 1
24–26 September 2026
- Sky's melodica upload died at encode: an overdub has two audio tracks, the encoder named one output. maybelle#133
- Reported as "failed at IPFS". It never reached IPFS — every failure inside that button reads as a storage failure.
- Justin's two uploads vanished entirely — no file, no diagnostics, nothing on the server. Unexplained.
- pickipedia#127: messages name the step that failed; the browser records uploads that die on its side; the panel renders without a token, which is why Justin saw a blank page.
- pickipedia#126: that repo had no CI, and 203 Python tests had never once been run. They pass.
- The Immutable Travel Logs built for Sky — phone recordings newest first, each under its own content address.
- hunter hit 89%. maybelle#134: tiered retention, cache prune, login warning. Simulated year: 282 dumps → 31, ~46 GB back.
- The backup mirror grows forever on purpose — hunter is the long-retention copy, so no
--delete. - Cryptograss:Video quality and storage: CRF 36 saves 24–34%, invisible in stills. Decided against it — all published video is ~2 GB; degrading masters to save half a gig is the wrong trade.
- Corrected my own figure: "~25 GB of unused images" is really 8.2 GB once shared layers are counted.[unverified]
Day 2
27 September 2026
- Justin: publishing only the first audio track throws away what somebody uploaded. He was right; the fix got replaced rather than defended.
- Every track is now an HLS alternate rendition — pick your take while watching. Audio encoded once, not once per video size.
- Found alongside it: silent videos could not publish at all. Same root cause, same PR.
- Justin merged #133, #134 and pickipedia#127 in eight minutes and redeployed delivery-kid.
- Melodica retried — failed again. Cause: an APAC track (Apple spatial audio); ffmpeg has no decoder and reports the codec as literally
none. - Sky asked whether the next fix was fully thought out. It wasn't — it decided by reading metadata, and a label is a guess.
- Replaced with a decode test: half a second of every track run through ffmpeg first. Pass, publish; fail, skip it and put ffmpeg's own words on the page. maybelle#136, merged and deployed the same evening.
- HDR check while waiting: nothing published is affected — all five releases are plain BT.709.
- But an HDR upload would publish as 8-bit pixels still labelled BT.2020 and PQ — looks flat, encode succeeds, nothing warns. maybelle#137 tone maps it properly.
- Reviewed #137 against the real image: I had tested on ffmpeg 8.0.1; we deploy Debian's 7.1.5. Chain works there.
- Corrected myself in public: HDR posters aren't missing on 7.1.5 as I'd claimed, just wrong.
- The serious catch: as first written, a failed tone map would have turned a flat-but-published video into a failed upload — the same mistake #136 existed to fix. Now probed first, falls back to publishing.
- Cost measured: ~2× encode time on 1080p (3.8s → 7.1s per ten seconds). No watchdog to trip.
- Melodica still unpublished — waiting on one deploy now, not on a bug.[unverified]
Still open
--rebuild — then the melodica take publishes