diff --git a/docs/v2/NOTES-FROM-POC.md b/docs/v2/NOTES-FROM-POC.md new file mode 100644 index 0000000..77bd2a1 --- /dev/null +++ b/docs/v2/NOTES-FROM-POC.md @@ -0,0 +1,142 @@ +# Notes for V2 from the POC thread (21–22 Oct 2026) + +What came out of a long working session on the POC that the V2 spec doesn't already say, or where the POC +experience suggests a different call. Read alongside the V2 handover spec and `HANDOFF.md` (its **Player rules** +section is the iOS lessons; they apply to any client). Nothing here is decided unless it says so. + +## Direction + +- **Leaning native-native** (Swift/SwiftUI first) over Expo. The reasons that matter for gandora specifically: + - **The DJ can run on the phone.** iOS 26 Foundation Models does guided generation into a Swift struct; the + struct is the sheet (`title, tags, lyrics, bpm, key, energy, seconds, meter, line`). Free, offline, ~1 s. + Rendering still goes to Runware (ACE-Step doesn't fit on a phone). + - **Testability.** Today's regressions all came from "the only iOS test rig is a phone". XCTest + simulator + exercise real AVFoundation: background audio, interruptions, lock screen, route changes. + - `AVQueuePlayer` pre-enqueues the next track, so most of the POC's hold/crackle/`ended` machinery goes away. +- Covers (`hash32`, `prng`, rings/chord/attractor) port line-for-line; keep integer ops identical for parity. +- The web POC stays as the lab, the Android/desktop fallback and the share-link landing. Features are invented + there and promoted once they've earned it (the spec's parity tracker). +- Alternatives considered: Capacitor (fastest path, wraps the current page), Tauri (early on mobile; good for a + desktop app with local ACE-Step), Flutter, Expo (right if the UI is being rebuilt anyway). + +## Backend + +- **Auth: Supabase Web3** (`signInWithWeb3`, Solana/Ethereum, EIP-4361). chai's call. + - Put the ToS acceptance in `statement` (Venice ToS §7.3 needs end users bound by terms). + - Wallets are free to mint, so **free tier is per device, not per account**: no login needed to listen; + login is for sync and paying. Enable per-IP rate limits + captcha. + - No recovery for wallet-only accounts: nudge `linkIdentity()` once sync is on. + - iOS PWA + wallets is clunky (deep links / in-wallet browser). Native solves it. + - No nostr provider; a custom-JWT edge function could add NIP-07 later. +- Minimal first slice: one Supabase project, tables `stations`, `tracks`, `events` with RLS on the wallet + identity, the existing Cloudflare worker verifying the Supabase JWT, blobs in R2. + +## Rendering: ACE-Step and Runware + +- ACE-Step 1.5 is **open weights, MIT** (code and weights). Model card: commercial use of outputs allowed; + training data claimed as licensed + royalty-free/public-domain + synthetic MIDI audio. +- **Runware already exposes repaint and extend** (schema: `runware.ai/docs/models/ace-step-v1-5-xl-turbo/schema.json`): + - `inputs.audio` (URL/UUID/data URI) + `settings.repaintingStart` / `repaintingEnd` (s) = repaint a section; + `repaintingEnd` past the end appends (extend), negative `repaintingStart` prepends (intro). + - `strength` 0–1 with `inputs.audio` = restyle; `numberResults` 1–4 = variations. + - No stems/extract/lego/cover task types on Runware; those need self-hosting. + - `duration` must be omitted when `inputs.audio` is set; `timeSignature` is beats (2/3/4/6), not "3/4". +- Product ideas from this: repaint around a reaction pill ("redo this bit"), ⟶ extend, restyle remixes. +- **Self-hosting is for customization, not cost** (Princess's priority). Runware turbo is $0.0045 per 2-min + track, about what a fully busy GPU costs. Self-hosting buys: LoRAs (per station, per DJ persona, from old + tracks), the extra task modes, the planner LM, XL without paying double. + - VRAM tiers (repo): 8–16 GB = 2B turbo + small LM; 20–24 GB = XL turbo without offload. + - LoRA training: ~1 h for 8 songs on a 12 GB GPU (repo figure). + - **Measured 22 Oct on hb-5 (M1, 8 GB):** fails to load. Even the smallest config (2B turbo, no LM) ran out of + GPU memory loading the VAE after the DiT (9.05 of 9.07 GiB allowed, 3 GB into swap). Realistic floor: a 16 GB + Mac for 2B turbo, 24 GB+ for comfort and near-XL, 36 GB+ for a Studio machine that also trains LoRAs; on GPUs + 8–12 GB for 2B turbo, ~20–24 GB for XL. For LoRA/stems experiments, rent a 24 GB GPU for an hour first. + +## Taste: embeddings direction + +- One multimodal space (Gemini Embedding 2, 768-d): seed, sheet, uploaded audio, rendered track all as points. + Station = region (centroid of kept, shadow of avoided); taste = cosine to that direction; far more + data-efficient than the 54→12→1 net on a few dozen reactions. +- DJ gets **nearest-neighbour exemplars** (RAG for the DJ) instead of `tasteText()`. +- Costs: text embeddings ~free; audio ~$0.01/min (≈4× a turbo render), so audio-embed uploads, kept tracks + and ~30 s around reaction ticks, not every track. Stamp vectors with model/dims/chunking. +- Later, owning the model: **gisting** (compress persona/rules into a few tokens) and **xRAG** (feed retriever + embeddings straight in as tokens) let a local DJ be conditioned on the taste vector directly. Needs an + open-weights DJ (Gemma-class) + a trained projector; every kept track is a training pair. + +## Product + +- **Market**: no shipping product combines per-listener *generated* songs + a talking DJ + learned taste. + Spotify DJ / Yoodio / Trending Music have the voice on licensed catalog; Endel / Noema generate but have no + voice; Suno Live Radio etc. are shared broadcasts. The window is cost-driven and won't stay open. +- **Pricing sketch** (not decided): Free (≈20 new tracks/day per device, then kept/catalog; noise/tones/ASMR + unlimited; DJ-read sponsor breaks) · Listener ~$7/mo (unlimited new music, soft cap ~3 h/day generation, + no sponsor breaks, all voices, sync) · Studio ~$18/mo (XL, remix/repaint/extend, downloads, own-voice DJ). + Price in time and tracks, never credits. Track **cost per kept track**. +- **Uploads / analyzer**: upload old tracks → local DSP (bpm, key, energy, sections) + Whisper lyrics + DJ + tags → an imported library row → station seeded from it with the file as reference audio. +- **Feedback button**: 🐛 files a Gitea issue with build rev, player state, current sheet, last ledger lines and + console errors attached; dedupe by fingerprint; the factory triages `feedback` issues. + +## Rights and moderation + +- Runware: outputs owned by the user; model licences are the user's responsibility. Venice: user keeps rights + in outputs; API products need end-user terms at least as protective; no third-party voice/likeness without + consent. Kokoro: Apache-2.0. +- US Copyright Office: purely AI-generated music isn't copyrightable. Don't promise exclusivity. +- **EU AI Act Art. 50** (in force 2 Aug 2026): mark synthetic audio machine-readably (at least ID3 / + C2PA-style tag from the worker) and disclose. Applies to EU listeners regardless of where we are. +- To do before anyone outside the team listens: ToS + AI disclosure; output tagging; written confirmation from + Runware / ACE Studio on commercial streaming; voice-clone policy (own sample + consent only). +- **Moderation**: check the sheet (tags, lyrics, DJ line) in the worker before render/TTS; strip real artist and + people names from tags; check persona text on save; report button on shared content; per-station explicit + setting (hard rules stay on). Venice is permissive, so the guard has to be ours. + +## DJ prompt optimisation: DSPy / GEPA (proposed, not started) + +- Treat the DJ as a DSPy program: station context + taste + recent reactions -> the sheet (strict schema). The + instructions to optimise are the next-track prompt, the break prompt, `MODE_BRIEF` and persona framing. +- **GEPA** fits because it learns from *textual* feedback via reflection on traces, and gandora has lots of it + (typed reactions, timed emojis, keeps/skips). It keeps a Pareto frontier of instructions, which matches + per-station taste (an ASMR-best prompt and a freestyle-best prompt both survive). +- Dataset: exported station histories (sheets + the reactions that followed; `stats.html` EXPORT JSON has most of it). +- Metric, two layers: (1) cheap checks: schema valid, avoid-list respected, no real artist/person names, variety + vs recent tracks; (2) an LLM judge for "does this sheet follow from these reactions", whose reasoning is the + feedback text GEPA reflects on. Human keep rate stays the final arbiter (judges can be gamed). +- Run offline in Python against the same Venice models via the worker; promote winners as pinned Langfuse prompt + versions (V2) or port back into the DJ prompt code (POC). Hold out stations; A/B on keep rate before switching. + +## Renderer economics: Runware vs self-hosted (RunPod), 22 Oct + +- Per 2-min turbo track: Runware $0.0045; RunPod serverless roughly the same when warm, worse with cold starts + (~30–60 s loading ~9 GB of weights); an always-on 4090 pod (~$0.34/hr community, ~$245/mo) only beats Runware + above ~75 tracks/hr sustained. XL roughly doubles both sides. +- Functionality is the real difference: Runware has text2music, reference audio, repaint/extend, strength, + variations; self-hosting adds LoRAs, stems/extract/lego/complete, full planner-LM and sampler control. +- Plan: Runware stays default. RunPod pod by the hour for LoRA training; a RunPod/Modal serverless endpoint only for + renders that need a LoRA or stems, selected by the worker from the sheet (e.g. `lora: "..."`). + +## Taste dynamics: measurement ideas (research, not product yet) + +- **Forward vs time-reversed training:** retrain the brain on a station's reactions in reverse order and compare + predictions; the gap = path dependence (how much is recency vs a stable summary). +- **Signed area of the taste path** (Lévy area of c(t), the station centroid over time): antisymmetric under time + reversal, so it separates a consistent drift/turn (non-zero) from wandering (cancels) and collapse to a default + (little area). Cheap enough to show as a curl indicator on the TASTE view. +- **Telephone loop:** sheet -> render -> (future) audio embedding -> DJ reads it back -> next sheet. Station drift + splits into noise (diffusive), attractor (collapse to generic, a bug) and holonomy (direction-dependent turn, + maybe a feature). Reversing loop order flips holonomy but not the other two. +- Context: a math thread on holonomy / Berry phase across embedding-space charts (xRAG projector as a transition + map; loop products of Procrustes maps between embedding models as discrete holonomy). Explainer video: + `teahubxyz/media` -> `manim/transition-map-narrated.mp4`. + +## POC housekeeping that affects V2 + +- **The module split is done on the `split` branch** (ES modules, no build step: `app.js core.js covers.js dj.js + render.js store.js player.js mashup.js ui/*.js`; `index.html` is markup + CSS). It is current with `main` apart + from these docs, every module parses, and the Safari harness gives the same results as `main`. Not yet merged: + it needs the dev → phone → main pass. It produces the pure pieces a Swift port reads from (covers math, sheet + contract, brain, DJ prompts). `docs/v2/REFACTOR-PROMPT.md` was written before we saw the branch and is + superseded by it. +- `tools/harness.sh safari` runs the loop in WebKit via safaridriver on hb-5 (needs `safaridriver --enable`). +- Player changes go `dev` → phone → `main` (HANDOFF.md). diff --git a/docs/v2/REFACTOR-PROMPT.md b/docs/v2/REFACTOR-PROMPT.md new file mode 100644 index 0000000..27853e7 --- /dev/null +++ b/docs/v2/REFACTOR-PROMPT.md @@ -0,0 +1,104 @@ +# Split gandora's index.html into ES modules — brief for a fresh agent + +> **Superseded.** The split already exists on the `split` branch (see `docs/v2/NOTES-FROM-POC.md`). Kept for its +> rules and checklist, which still describe how to review that branch before it merges. + +You are refactoring **gandora/site** on teahub.xyz (Gitea; use the Teahub MCP tools: `tea_file_get`, `tea_file_edit`, `tea_file_change`, `tea_commit`, `tea_branch_create`, `tea_pull_create`; there is no git access from the container). The Mac **hb-5** is reachable through `boring_mac_exec` (120 s per call; long jobs with `nohup … &` then poll); a clone lives at `~/scratch/gsite` (`git pull -q` to refresh). Node is at `/opt/homebrew/bin/node`. + +Work on the **`dev` branch only.** Do not push to `main`. Open one PR at the end. + +## Goal + +`index.html` is a ~2,100-line single file holding the player, the DJ, rendering, covers, storage, the UI sheets and the usage ledger in one scope. Split it into ES modules **with no build step**: plain `