docs: README (what it is, how it runs), ARCHITECTURE (every moving part, the playback state machine, the data), HANDOFF (seams, known rough edges, what a refactor should keep); tools/harness.html as the playback regression test
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K6897PWcUEwHiqZMcVnNF6
This commit is contained in:
1 parent
2dbbbbe2be
commit
4a33a30820
5 files changed
+221
No files matched your search
@@ -0,0 +1,77 @@
|
|||||||
|
# How g*ndora fits together
|
||||||
|
|
||||||
|
One page, one script, one worker. This is the map: what each part owns, how a track moves through the system, and
|
||||||
|
the invariants that the playback code relies on. Keep it current when you change any of them.
|
||||||
|
|
||||||
|
## The parts
|
||||||
|
|
||||||
|
| Part | Owns | Where |
|
||||||
|
|---|---|---|
|
||||||
|
| Seed screen | the seed box, the stations row, LIBRARY / TASTE / LIGHT / RELOAD | `#seed`, `listStations`, `openStation` |
|
||||||
|
| Station | seed, name, taste {like, avoid, notes}, history (last 8 sheets), look, palette, DJ name + voice, `welcomed` | `openStation`, `saveStation`, `stationSheet`; IndexedDB `stations` |
|
||||||
|
| DJ | writes a sheet (JSON, `DJ_SCHEMA`) from seed + taste + recent tracks + the brain's taste text | `djPrompt` (base) + brain.js override, `askDJ`; breaks: `askBreak`; mashups: `askMash` |
|
||||||
|
| Renderer | audio from a sheet | `renderTrack` → worker `/v1/render` (Runware; seed, bpm, key, meter, reference audio) or Venice queue/poll with a personal key; spoken breaks → `speak` → `/v1/speech` |
|
||||||
|
| Player | one `<audio>` (`state.audio`), play/pause/skip/back/replay/loop/shuffle/remix, scrubbing, Media Session | `playJob`, `advance`, `step`, `swapNow`, the `ended` handler |
|
||||||
|
| Room tone | the vinyl crackle: a second `<audio>` looping a generated 4 s wav whenever music isn't playing; also used *inside the player* as the hold between tracks | `makeCrackle`, `crackleOn/Off`, `bed`, `swapNow` |
|
||||||
|
| Covers | drawn from the sheet: hue from key, saturation from energy, shape from seed; looks rings / chord / attractor; palette mono / duo / tri; breaks get the ON AIR treatment | `drawCover`, `palette`, `drawMark`, `drawOnAir`; visualizer `vizStart` (analyser) / `vizBeat` (iOS) |
|
||||||
|
| Reactions | emoji / text / tag taps → taste edits on the station + `spec.reaction` string + timed `spec.hits`; brain retrains | `react`, `tagTap`, `markReaction`, brain.js `react` override |
|
||||||
|
| Breaks | every `BREAK_EVERY` slots once the station has 3 real tracks: welcome (once), ident, recap, shout-out, tease, sponsor | `askBreak`, `BREAKS`, `cookNext` |
|
||||||
|
| Library | every rendered track (unkept ones roll off after `KEEP_UNKEPT`), KEPT, grouped by station, tapes strip, now-playing bar | `db`, `remember`, the library render, `libNow` |
|
||||||
|
| Tapes | named ordered lists of track ids; KEPT is a virtual tape; cover from `tapeSpec`; MASHUP | `openTape`, `playTape`, `pickTape`, `tapeSpec`, `mashup` |
|
||||||
|
| Taste | per-station + overall nets over (tempo, energy, key x/y, minor, length, 48 hashed word slots); THE FIVE; map; brain drawing; `% YOU` | brain.js |
|
||||||
|
| Links | `#seed` follows the station; `?t=` carries a sheet (+ render seed, length, meter, look, palette); `?tape=` a tape | `setUrl`, `shareLink`, the boot block at the end |
|
||||||
|
| PWA | manifest, icons, `sw.js` (network-first, shell cached), update pill, RELOAD | head, `sw.js`, the updates block |
|
||||||
|
|
||||||
|
## A track's life
|
||||||
|
|
||||||
|
1. `cookNext()` asks the DJ (`askDJ`, or `askBreak` when a break is due) → a `spec`. `makeJob(spec)` starts the render
|
||||||
|
immediately and returns `{ spec, blob: null, done: Promise }`; the brain adds `spec.guess`.
|
||||||
|
2. `advance()` picks what plays next: tape → queued shared tape → forward in session (`step(+1)`) → the remix/after
|
||||||
|
slot → shuffle (maybe) → `state.nextJob` → or cooks one now.
|
||||||
|
3. `playJob(job)` sets the UI from the sheet, waits for `job.done`, sets `state.audio.src`, starts `cookNext()` for the
|
||||||
|
one after, calls `play()`, then does the bookkeeping (`remember` → IndexedDB, `saveStation`, trail).
|
||||||
|
4. On `ended`: `swapNow()` — if the next job is cooked and nothing is queued ahead of it, its source starts **inside the
|
||||||
|
event, synchronously** (iOS only keeps background playback alive that way). If nothing is cooked, the player itself
|
||||||
|
loops the crackle (`state.holding`); the landing track is queued (`state.pendingJob`) and takes over at the end of
|
||||||
|
the crackle's pass, again inside `ended`. Then `advance()` runs the bookkeeping.
|
||||||
|
|
||||||
|
## Playback invariants (the things that have broken before)
|
||||||
|
|
||||||
|
- **Unlock.** The first pointer/key event plays a 50 ms silent wav (`SILENCE`, built in code) in `state.audio` and starts
|
||||||
|
the crackle element; `state.primed` is set when that play resolves. Every later `play()` rides on it. The `ended`
|
||||||
|
handler must ignore the unlock clip.
|
||||||
|
- **Nothing awaits before `play()` except the render.** No database reads between setting `src` and `play()`.
|
||||||
|
`db.get(id)` for one row; `db.all()` loads every track *with its audio* — never use it to find one row.
|
||||||
|
- **Crackle never routes through the player on desktop** except as the hold. On iOS the analyser is never attached
|
||||||
|
(`IOS` → `vizBeat`), so the music stays a plain element (background playback, mute switch).
|
||||||
|
- **The bed is a pure function of state** (`bed()`: not playing, not holding, player paused → on), debounced 300 ms,
|
||||||
|
fades in over 600 ms. Call `bed()` on state changes; never toggle the crackle directly from UI code.
|
||||||
|
- **Source swaps that must survive a locked screen happen synchronously inside `ended`.** Everything async goes after.
|
||||||
|
- **Breaks never train taste** (`spec.break`), never shuffle, never seed THE FIVE. Spoken breaks are TTS (`spec.spoken`).
|
||||||
|
|
||||||
|
## The worker (teahubxyz/time › .github/time/cloudflare/inference.js)
|
||||||
|
|
||||||
|
`x-app: gandora` from an allowed origin. Routes: `/v1/chat/completions` (Venice, CHAT_MODELS), `/v1/render`
|
||||||
|
(Runware ACE-Step: prompt, lyrics, duration ≤ AUDIO_SECONDS=300, bpm, key, timeSignature as beats 2/3/4/6, seed,
|
||||||
|
quality, `reference` data-URL ≤ 8 MB which also fixes the length; Venice fallback; headers x-cost / x-seed /
|
||||||
|
x-renderer / x-fallback), `/v1/speech` (Venice TTS), `/v1/audio/*` and `/v1/image/generate` (Venice pass-through).
|
||||||
|
Quota: one KV write per call, per person per day; fails open. Deployed by Actions › inference on push.
|
||||||
|
|
||||||
|
## Data
|
||||||
|
|
||||||
|
See STORAGE.md. In short: `tracks` {id, spec, blob, made, kept, plays}; `stations` {id, seed, name, taste, history,
|
||||||
|
look, palette, djName, djVoice, welcomed, played, last}; `tapes` {id, name, ids, made, last}. The sheet (`spec`) is
|
||||||
|
the unit of everything: title, tags, lyrics, bpm, key, energy, seconds, meter, line, seed, stationId, made, id,
|
||||||
|
rseed (render seed), cost, renderer, reaction, hits, guess, break/sponsored/spoken, remixOf, mashOf, look, palette.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
`tools/harness.html` loads the live page with the backend stubbed (DJ returns a canned sheet, render returns a bundled
|
||||||
|
mp3 after a configurable delay) and drives the player through: first play, pre-cook, handoff on end, the crackle
|
||||||
|
hold and its handover, pause/resume, skip. It prints one state line per step. Run it in headless Chrome:
|
||||||
|
|
||||||
|
sh tools/harness.sh # builds the page from index.html, runs Chrome, prints the state lines
|
||||||
|
|
||||||
|
Every line should show the expected `cur`/`next`/`paused`/`holding`. Run it before and after any change to playback,
|
||||||
|
the crackle, the hold, or `playJob`/`advance`. It cannot test real audio output, autoplay policy, iOS, or the
|
||||||
|
OfflineAudioContext used by mashups — those need a phone.
|
||||||
+44
@@ -0,0 +1,44 @@
|
|||||||
|
# Handoff notes
|
||||||
|
|
||||||
|
For whoever picks this up: what is solid, what is held together with care, and what a refactor should and should not
|
||||||
|
do. Read ARCHITECTURE.md first.
|
||||||
|
|
||||||
|
## Where the seams are
|
||||||
|
|
||||||
|
- `index.html` is ~1,900 lines in one script. It grew feature by feature in one long session. The sections are
|
||||||
|
commented and ordered sensibly, but state is one global `state` object and functions reach into each other freely.
|
||||||
|
- brain.js **overrides** page functions by assignment (`djPrompt = …`, `openStation = …`, `makeJob = …`,
|
||||||
|
`react = …`) so it can be loaded after the page. A refactor should turn these into explicit hooks.
|
||||||
|
- Playback has three overlapping mechanisms layered over time: the ordinary `playJob/advance` path, the synchronous
|
||||||
|
`ended` swap (`swapNow`) for locked screens, and the crackle hold with `pendingJob`. They work together (the
|
||||||
|
harness proves the paths) but they should be one small state machine: `idle → cooking → holding → playing → ended`.
|
||||||
|
- The crackle exists twice: as the side element (room tone) and as the player's own source (the hold). That is
|
||||||
|
deliberate (iOS needs the hold in the same element) but easy to misread.
|
||||||
|
- `db.all()` is used in many places where `db.get` or a filtered index would do; it loads every blob.
|
||||||
|
- Several things are computed from the sheet in more than one place (section-at-time, mmss, kind chips).
|
||||||
|
|
||||||
|
## Known rough edges
|
||||||
|
|
||||||
|
- iOS volume sliders are inert (platform); the welcome/ident may fall back to Venice if Runware times out on a
|
||||||
|
spoken-tag render (now avoided: spoken breaks are TTS).
|
||||||
|
- The mashup reference uses OfflineAudioContext; untested headlessly.
|
||||||
|
- The quota is per-person-per-day only; KV free tier caps total writes at ~1,000/day.
|
||||||
|
- Share links made before the render seed was included re-roll the song.
|
||||||
|
- `KEEP_UNKEPT = 40` unkept tracks roll off; nothing warns.
|
||||||
|
- Light theme: a few hard-coded colours may remain in rarely-seen sheets.
|
||||||
|
|
||||||
|
## What a refactor should keep
|
||||||
|
|
||||||
|
- Every invariant in ARCHITECTURE.md › Playback invariants, and the harness green before and after.
|
||||||
|
- The sheet as the unit (everything derives from `spec`), covers as a pure function of it.
|
||||||
|
- One static page, no build step, no framework, no keys in the page.
|
||||||
|
- STORAGE.md's contract (chai's DB adapter targets it).
|
||||||
|
|
||||||
|
## What's next (agreed directions, not started)
|
||||||
|
|
||||||
|
- Text embeddings for every sheet, audio embeddings (Gemini Embedding 2) for kept/reacted tracks; the taste map
|
||||||
|
laid out in embedding space; segment-level vectors with the timed reactions (`spec.hits`).
|
||||||
|
- Free tier = shared allowance on the worker (the key box is parked); pro = longer tracks, XL renders, sync.
|
||||||
|
- Room-tone picker (vinyl / rain / tape / brown / drone in key).
|
||||||
|
- Butterchurn (Milkdrop) as a fourth look.
|
||||||
|
- Taste file export/import.
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# g*ndora
|
||||||
|
|
||||||
|
A radio station that writes its own songs. Type a seed, get a station: a DJ (an LLM) writes each track's sheet music
|
||||||
|
(title, tags, lyrics, bpm, key, length, meter, a line to say), a song model (ACE-Step) renders it, the cover is drawn
|
||||||
|
from the song's own settings, and every reaction you make teaches a small taste model that steers the DJ.
|
||||||
|
|
||||||
|
Live at https://gandora.lol (also gandora.yolopreneur.lol). Installable as a PWA; the library plays offline.
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
It is one static page. Serve the folder and open it:
|
||||||
|
|
||||||
|
python3 -m http.server 8794 && open http://127.0.0.1:8794/
|
||||||
|
|
||||||
|
It talks to the factory's inference worker at `https://inference.yolopreneur.lol` (allowed origins include
|
||||||
|
localhost? no: only `*.yolopreneur.lol` and `gandora.lol`; for local work add your Venice key with `/?key` or run
|
||||||
|
from a `*.yolopreneur.lol` host). Everything the listener makes lives in their browser (IndexedDB) — see STORAGE.md.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `index.html` — the whole app: markup, styles, and the player, station, library, tapes, breaks, covers, sheets.
|
||||||
|
- `brain.js` — the taste model (a small net per station and overall), the TASTE screen, the DJ's taste text.
|
||||||
|
- `sw.js` — service worker: caches the shell; stamped with the commit by the shots workflow.
|
||||||
|
- `manifest.json`, `icons/`, `favicon.svg` — PWA. Icons are drawn by `tools/icons.py` (stdlib only).
|
||||||
|
- `tools/shots.sh` — the card image and the icons, run by `.gitea/workflows/shots.yml` on every push.
|
||||||
|
- `tools/harness.html` — the playback regression harness (see ARCHITECTURE.md › Testing).
|
||||||
|
- `STORAGE.md` — the IndexedDB contract. `ARCHITECTURE.md` — how it all fits. `HANDOFF.md` — read before changing things.
|
||||||
|
|
||||||
|
## Backend
|
||||||
|
|
||||||
|
The page never holds a key. `inference.yolopreneur.lol` (teahubxyz/time, `.github/time/cloudflare/inference.js`)
|
||||||
|
routes per capability: chat → Venice (claude-haiku), render → Runware ACE-Step (Venice as fallback), speech → Venice
|
||||||
|
TTS. Secrets and deploys: `.github/time/SECRETS.md` in that repo.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<title>harness</title>
|
||||||
|
<p>This file is a fragment: tools/harness.sh appends everything from the marker below to index.html and runs it.</p>
|
||||||
|
<!-- harness -->
|
||||||
|
<script>
|
||||||
|
try {
|
||||||
|
const MP3 = "data:audio/mpeg;base64,__MP3__";
|
||||||
|
async function mp3blob() { const b = atob(MP3.split(",")[1]); const u = new Uint8Array(b.length); for (let i = 0; i < b.length; i++) u[i] = b.charCodeAt(i); return new Blob([u], { type: "audio/mpeg" }); }
|
||||||
|
let renderDelay = 50, n = 0;
|
||||||
|
venice = async function (path, body, { raw = false } = {}) {
|
||||||
|
if (path.includes("chat")) return { choices: [{ message: { content: JSON.stringify({ title: "T" + (++n), tags: "lo-fi, piano, instrumental", lyrics: "[verse]\nla\n[chorus]\nlo", bpm: 80, key: "C major", energy: 0.4, seconds: 30, meter: "4/4", station: "Test FM", line: "hi" }) } }] };
|
||||||
|
if (path === "/render") { await new Promise((r) => setTimeout(r, renderDelay)); const b = await mp3blob(); return new Response(b, { headers: { "content-type": "audio/mpeg", "x-cost": "0.001", "x-renderer": "test" } }); }
|
||||||
|
throw new Error("unexpected " + path);
|
||||||
|
};
|
||||||
|
const log = []; const dd = document.createElement("pre"); dd.id = "diag"; document.body.append(dd);
|
||||||
|
const L = (m) => { log.push(m); dd.textContent = log.join("\n"); };
|
||||||
|
const S = (tag) => L(tag + ": src=" + String(state.audio.src).slice(5, 14) + " paused=" + state.audio.paused + " playing=" + state.playing + " holding=" + !!state.holding + " cur=" + (state.current && state.current.spec.title) + " next=" + (state.nextJob && state.nextJob.spec.title) + " nextblob=" + !!(state.nextJob && state.nextJob.blob) + " crackle=" + crackle.on + " loop=" + state.audio.loop);
|
||||||
|
window.addEventListener("error", (e) => L("ERR " + e.message + " @" + e.lineno));
|
||||||
|
window.addEventListener("unhandledrejection", (e) => L("REJ " + (e.reason && (e.reason.stack || e.reason)).toString().slice(0, 200)));
|
||||||
|
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
|
||||||
|
const ended = () => { state.audio.dispatchEvent(new Event("ended")); };
|
||||||
|
(async () => {
|
||||||
|
document.dispatchEvent(new Event("keydown")); await sleep(300);
|
||||||
|
openStation("test seed"); await sleep(1500); S("after first (expect cur=T1 playing, next=T2 cooked)");
|
||||||
|
ended(); await sleep(600); S("after ended (expect cur=T2 playing, holding=false)");
|
||||||
|
renderDelay = 3000; state.nextJob = null; cookNext(); await sleep(300); S("slow cook started (expect next cooking, nextblob=false)");
|
||||||
|
ended(); await sleep(600); S("ended, nothing ready (expect holding=true, loop=true)");
|
||||||
|
await sleep(3500); S("track landed (expect holding=true, loop=false: queued behind the crackle)");
|
||||||
|
ended(); await sleep(600); S("crackle pass ended (expect cur=next track playing, holding=false)");
|
||||||
|
renderDelay = 50; document.querySelector("#play").click(); await sleep(600); S("after pause (expect paused, crackle=true)");
|
||||||
|
document.querySelector("#play").click(); await sleep(300); S("after resume (expect playing, crackle=false)");
|
||||||
|
document.querySelector("#skip").click(); await sleep(1500); S("after skip (expect a new cur)");
|
||||||
|
L("END");
|
||||||
|
})();
|
||||||
|
} catch (e) { const p = document.createElement("pre"); p.id = "diag"; p.textContent = "TOPERR " + e.message; document.body.append(p); }
|
||||||
|
</script>
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# tools/harness.sh: the playback regression run. Builds tools/harness.html's page from index.html, runs it in headless
|
||||||
|
# Chrome with autoplay allowed, prints one state line per step. Needs Chrome/Chromium and python3.
|
||||||
|
set -eu
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
CHROME=${CHROME:-}
|
||||||
|
for c in "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" "$(command -v chromium || true)" "$(command -v google-chrome || true)"; do
|
||||||
|
if [ -z "$CHROME" ] && [ -n "$c" ] && [ -x "$c" ]; then CHROME=$c; fi
|
||||||
|
done
|
||||||
|
[ -n "$CHROME" ] || { echo "no Chrome found; set CHROME" >&2; exit 1; }
|
||||||
|
out=$(mktemp -d)
|
||||||
|
python3 - "$out" <<'EOF'
|
||||||
|
import sys, pathlib
|
||||||
|
out = pathlib.Path(sys.argv[1])
|
||||||
|
page = pathlib.Path("index.html").read_text()
|
||||||
|
h = pathlib.Path("tools/harness.html").read_text()
|
||||||
|
script = h[h.index("<!-- harness -->"):]
|
||||||
|
mp3 = pathlib.Path("tools/harness.mp3").read_bytes() if pathlib.Path("tools/harness.mp3").exists() else b""
|
||||||
|
import base64
|
||||||
|
script = script.replace("__MP3__", base64.b64encode(mp3).decode())
|
||||||
|
(out / "run.html").write_text(page.replace("</body>", script + "</body>"))
|
||||||
|
EOF
|
||||||
|
"$CHROME" --headless=new --disable-gpu --autoplay-policy=no-user-gesture-required --virtual-time-budget=25000 --dump-dom "file://$out/run.html" 2>/dev/null > "$out/dom.html" || true
|
||||||
|
python3 - "$out/dom.html" <<'EOF'
|
||||||
|
import sys, re, html
|
||||||
|
s = open(sys.argv[1]).read()
|
||||||
|
m = re.search(r'<pre id="diag">(.*?)</pre>', s, re.S)
|
||||||
|
print(html.unescape(m.group(1)) if m else "no diag: the harness did not run")
|
||||||
|
EOF
|
||||||
|
rm -rf "$out"
|
||||||
Reference in new issue
Block a user