site/docs/v2/NOTES-FROM-POC.md

Notes for V2 from the POC thread (21–22 Oct 2026)

What came out of a long working session on the POC that the V2 spec doesn't already say, or where the POC experience suggests a different call. Read alongside the V2 handover spec and HANDOFF.md (its Player rules section is the iOS lessons; they apply to any client). Nothing here is decided unless it says so.

Direction

• Leaning native-native (Swift/SwiftUI first) over Expo. The reasons that matter for gandora specifically:
  • The DJ can run on the phone. iOS 26 Foundation Models does guided generation into a Swift struct; the struct is the sheet (title, tags, lyrics, bpm, key, energy, seconds, meter, line). Free, offline, ~1 s. Rendering still goes to Runware (ACE-Step doesn't fit on a phone).
  • Testability. Today's regressions all came from "the only iOS test rig is a phone". XCTest + simulator exercise real AVFoundation: background audio, interruptions, lock screen, route changes.
  • AVQueuePlayer pre-enqueues the next track, so most of the POC's hold/crackle/ended machinery goes away.
• Covers (hash32, prng, rings/chord/attractor) port line-for-line; keep integer ops identical for parity.
• The web POC stays as the lab, the Android/desktop fallback and the share-link landing. Features are invented there and promoted once they've earned it (the spec's parity tracker).
• Alternatives considered: Capacitor (fastest path, wraps the current page), Tauri (early on mobile; good for a desktop app with local ACE-Step), Flutter, Expo (right if the UI is being rebuilt anyway).

Backend

• Auth: Supabase Web3 (signInWithWeb3, Solana/Ethereum, EIP-4361). chai's call.
  • Put the ToS acceptance in statement (Venice ToS §7.3 needs end users bound by terms).
  • Wallets are free to mint, so free tier is per device, not per account: no login needed to listen; login is for sync and paying. Enable per-IP rate limits + captcha.
  • No recovery for wallet-only accounts: nudge linkIdentity() once sync is on.
  • iOS PWA + wallets is clunky (deep links / in-wallet browser). Native solves it.
  • No nostr provider; a custom-JWT edge function could add NIP-07 later.
• Minimal first slice: one Supabase project, tables stations, tracks, events with RLS on the wallet identity, the existing Cloudflare worker verifying the Supabase JWT, blobs in R2.

Rendering: ACE-Step and Runware

• ACE-Step 1.5 is open weights, MIT (code and weights). Model card: commercial use of outputs allowed; training data claimed as licensed + royalty-free/public-domain + synthetic MIDI audio.
• Runware already exposes repaint and extend (schema: runware.ai/docs/models/ace-step-v1-5-xl-turbo/schema.json):
  • inputs.audio (URL/UUID/data URI) + settings.repaintingStart / repaintingEnd (s) = repaint a section; repaintingEnd past the end appends (extend), negative repaintingStart prepends (intro).
  • strength 0–1 with inputs.audio = restyle; numberResults 1–4 = variations.
  • No stems/extract/lego/cover task types on Runware; those need self-hosting.
  • duration must be omitted when inputs.audio is set; timeSignature is beats (2/3/4/6), not "3/4".
• Product ideas from this: repaint around a reaction pill ("redo this bit"), ⟶ extend, restyle remixes.
• Self-hosting is for customization, not cost (Princess's priority). Runware turbo is $0.0045 per 2-min track, about what a fully busy GPU costs. Self-hosting buys: LoRAs (per station, per DJ persona, from old tracks), the extra task modes, the planner LM, XL without paying double.
  • VRAM tiers (repo): 8–16 GB = 2B turbo + small LM; 20–24 GB = XL turbo without offload.
  • LoRA training: ~1 h for 8 songs on a 12 GB GPU (repo figure).
  • Measured 22 Oct on hb-5 (M1, 8 GB): fails to load. Even the smallest config (2B turbo, no LM) ran out of GPU memory loading the VAE after the DiT (9.05 of 9.07 GiB allowed, 3 GB into swap). Realistic floor: a 16 GB Mac for 2B turbo, 24 GB+ for comfort and near-XL, 36 GB+ for a Studio machine that also trains LoRAs; on GPUs 8–12 GB for 2B turbo, ~20–24 GB for XL. For LoRA/stems experiments, rent a 24 GB GPU for an hour first.

Taste: embeddings direction

• One multimodal space (Gemini Embedding 2, 768-d): seed, sheet, uploaded audio, rendered track all as points. Station = region (centroid of kept, shadow of avoided); taste = cosine to that direction; far more data-efficient than the 54→12→1 net on a few dozen reactions.
• DJ gets nearest-neighbour exemplars (RAG for the DJ) instead of tasteText().
• Costs: text embeddings ~free; audio ~$0.01/min (≈4× a turbo render), so audio-embed uploads, kept tracks and ~30 s around reaction ticks, not every track. Stamp vectors with model/dims/chunking.
• Later, owning the model: gisting (compress persona/rules into a few tokens) and xRAG (feed retriever embeddings straight in as tokens) let a local DJ be conditioned on the taste vector directly. Needs an open-weights DJ (Gemma-class) + a trained projector; every kept track is a training pair.

Product

• Market: no shipping product combines per-listener generated songs + a talking DJ + learned taste. Spotify DJ / Yoodio / Trending Music have the voice on licensed catalog; Endel / Noema generate but have no voice; Suno Live Radio etc. are shared broadcasts. The window is cost-driven and won't stay open.
• Pricing sketch (not decided): Free (≈20 new tracks/day per device, then kept/catalog; noise/tones/ASMR unlimited; DJ-read sponsor breaks) · Listener ~$7/mo (unlimited new music, soft cap ~3 h/day generation, no sponsor breaks, all voices, sync) · Studio ~$18/mo (XL, remix/repaint/extend, downloads, own-voice DJ). Price in time and tracks, never credits. Track cost per kept track.
• Uploads / analyzer: upload old tracks → local DSP (bpm, key, energy, sections) + Whisper lyrics + DJ tags → an imported library row → station seeded from it with the file as reference audio.
• Feedback button: 🐛 files a Gitea issue with build rev, player state, current sheet, last ledger lines and console errors attached; dedupe by fingerprint; the factory triages feedback issues.

Rights and moderation

• Runware: outputs owned by the user; model licences are the user's responsibility. Venice: user keeps rights in outputs; API products need end-user terms at least as protective; no third-party voice/likeness without consent. Kokoro: Apache-2.0.
• US Copyright Office: purely AI-generated music isn't copyrightable. Don't promise exclusivity.
• EU AI Act Art. 50 (in force 2 Aug 2026): mark synthetic audio machine-readably (at least ID3 / C2PA-style tag from the worker) and disclose. Applies to EU listeners regardless of where we are.
• To do before anyone outside the team listens: ToS + AI disclosure; output tagging; written confirmation from Runware / ACE Studio on commercial streaming; voice-clone policy (own sample + consent only).
• Moderation: check the sheet (tags, lyrics, DJ line) in the worker before render/TTS; strip real artist and people names from tags; check persona text on save; report button on shared content; per-station explicit setting (hard rules stay on). Venice is permissive, so the guard has to be ours.

DJ prompt optimisation: DSPy / GEPA (proposed, not started)

• Treat the DJ as a DSPy program: station context + taste + recent reactions -> the sheet (strict schema). The instructions to optimise are the next-track prompt, the break prompt, MODE_BRIEF and persona framing.
• GEPA fits because it learns from textual feedback via reflection on traces, and gandora has lots of it (typed reactions, timed emojis, keeps/skips). It keeps a Pareto frontier of instructions, which matches per-station taste (an ASMR-best prompt and a freestyle-best prompt both survive).
• Dataset: exported station histories (sheets + the reactions that followed; stats.html EXPORT JSON has most of it).
• Metric, two layers: (1) cheap checks: schema valid, avoid-list respected, no real artist/person names, variety vs recent tracks; (2) an LLM judge for "does this sheet follow from these reactions", whose reasoning is the feedback text GEPA reflects on. Human keep rate stays the final arbiter (judges can be gamed).
• Run offline in Python against the same Venice models via the worker; promote winners as pinned Langfuse prompt versions (V2) or port back into the DJ prompt code (POC). Hold out stations; A/B on keep rate before switching.

Renderer economics: Runware vs self-hosted (RunPod), 22 Oct

• Per 2-min turbo track: Runware $0.0045; RunPod serverless roughly the same when warm, worse with cold starts (~30–60 s loading 9 GB of weights); an always-on 4090 pod ($0.34/hr community, ~$245/mo) only beats Runware above ~75 tracks/hr sustained. XL roughly doubles both sides.
• Functionality is the real difference: Runware has text2music, reference audio, repaint/extend, strength, variations; self-hosting adds LoRAs, stems/extract/lego/complete, full planner-LM and sampler control.
• Plan: Runware stays default. RunPod pod by the hour for LoRA training; a RunPod/Modal serverless endpoint only for renders that need a LoRA or stems, selected by the worker from the sheet (e.g. lora: "...").

Taste dynamics: measurement ideas (research, not product yet)

• Forward vs time-reversed training: retrain the brain on a station's reactions in reverse order and compare predictions; the gap = path dependence (how much is recency vs a stable summary).
• Signed area of the taste path (Lévy area of c(t), the station centroid over time): antisymmetric under time reversal, so it separates a consistent drift/turn (non-zero) from wandering (cancels) and collapse to a default (little area). Cheap enough to show as a curl indicator on the TASTE view.
• Telephone loop: sheet -> render -> (future) audio embedding -> DJ reads it back -> next sheet. Station drift splits into noise (diffusive), attractor (collapse to generic, a bug) and holonomy (direction-dependent turn, maybe a feature). Reversing loop order flips holonomy but not the other two.
• Context: a math thread on holonomy / Berry phase across embedding-space charts (xRAG projector as a transition map; loop products of Procrustes maps between embedding models as discrete holonomy). Explainer video: teahubxyz/media -> manim/transition-map-narrated.mp4.

POC housekeeping that affects V2

• The module split is done on the split branch (ES modules, no build step: app.js core.js covers.js dj.js render.js store.js player.js mashup.js ui/*.js; index.html is markup + CSS). It is current with main apart from these docs, every module parses, and the Safari harness gives the same results as main. Not yet merged: it needs the dev → phone → main pass. It produces the pure pieces a Swift port reads from (covers math, sheet contract, brain, DJ prompts). docs/v2/REFACTOR-PROMPT.md was written before we saw the branch and is superseded by it.
• tools/harness.sh safari runs the loop in WebKit via safaridriver on hb-5 (needs safaridriver --enable).
• Player changes go dev → phone → main (HANDOFF.md).