site/docs/v2/NOTES-FROM-POC.md

Notes for V2 from the POC thread (21–22 Oct 2026)

What came out of a long working session on the POC that the V2 spec doesn't already say, or where the POC experience suggests a different call. Read alongside the V2 handover spec and HANDOFF.md (its Player rules section is the iOS lessons; they apply to any client). Nothing here is decided unless it says so.

Direction

• Leaning native-native (Swift/SwiftUI first) over Expo. The reasons that matter for gandora specifically:
  • The DJ can run on the phone. iOS 26 Foundation Models does guided generation into a Swift struct; the struct is the sheet (title, tags, lyrics, bpm, key, energy, seconds, meter, line). Free, offline, ~1 s. Rendering still goes to Runware (ACE-Step doesn't fit on a phone).
  • Testability. Today's regressions all came from "the only iOS test rig is a phone". XCTest + simulator exercise real AVFoundation: background audio, interruptions, lock screen, route changes.
  • AVQueuePlayer pre-enqueues the next track, so most of the POC's hold/crackle/ended machinery goes away.
• Covers (hash32, prng, rings/chord/attractor) port line-for-line; keep integer ops identical for parity.
• The web POC stays as the lab, the Android/desktop fallback and the share-link landing. Features are invented there and promoted once they've earned it (the spec's parity tracker).
• Alternatives considered: Capacitor (fastest path, wraps the current page), Tauri (early on mobile; good for a desktop app with local ACE-Step), Flutter, Expo (right if the UI is being rebuilt anyway).

Backend

• Auth: Supabase Web3 (signInWithWeb3, Solana/Ethereum, EIP-4361). chai's call.
  • Put the ToS acceptance in statement (Venice ToS §7.3 needs end users bound by terms).
  • Wallets are free to mint, so free tier is per device, not per account: no login needed to listen; login is for sync and paying. Enable per-IP rate limits + captcha.
  • No recovery for wallet-only accounts: nudge linkIdentity() once sync is on.
  • iOS PWA + wallets is clunky (deep links / in-wallet browser). Native solves it.
  • No nostr provider; a custom-JWT edge function could add NIP-07 later.
• Minimal first slice: one Supabase project, tables stations, tracks, events with RLS on the wallet identity, the existing Cloudflare worker verifying the Supabase JWT, blobs in R2.

Rendering: ACE-Step and Runware

• ACE-Step 1.5 is open weights, MIT (code and weights). Model card: commercial use of outputs allowed; training data claimed as licensed + royalty-free/public-domain + synthetic MIDI audio.
• Runware already exposes repaint and extend (schema: runware.ai/docs/models/ace-step-v1-5-xl-turbo/schema.json):
  • inputs.audio (URL/UUID/data URI) + settings.repaintingStart / repaintingEnd (s) = repaint a section; repaintingEnd past the end appends (extend), negative repaintingStart prepends (intro).
  • strength 0–1 with inputs.audio = restyle; numberResults 1–4 = variations.
  • No stems/extract/lego/cover task types on Runware; those need self-hosting.
  • duration must be omitted when inputs.audio is set; timeSignature is beats (2/3/4/6), not "3/4".
• Product ideas from this: repaint around a reaction pill ("redo this bit"), ⟶ extend, restyle remixes.
• Self-hosting is for customization, not cost (Princess's priority). Runware turbo is $0.0045 per 2-min track, about what a fully busy GPU costs. Self-hosting buys: LoRAs (per station, per DJ persona, from old tracks), the extra task modes, the planner LM, XL without paying double.
  • VRAM tiers (repo): 8–16 GB = 2B turbo + small LM; 20–24 GB = XL turbo without offload.
  • LoRA training: ~1 h for 8 songs on a 12 GB GPU (repo figure).
  • A benchmark of 2B turbo on hb-5 (M1, 8 GB, MLX) was started 21 Oct; result pending at the time of writing.

Taste: embeddings direction

• One multimodal space (Gemini Embedding 2, 768-d): seed, sheet, uploaded audio, rendered track all as points. Station = region (centroid of kept, shadow of avoided); taste = cosine to that direction; far more data-efficient than the 54→12→1 net on a few dozen reactions.
• DJ gets nearest-neighbour exemplars (RAG for the DJ) instead of tasteText().
• Costs: text embeddings ~free; audio ~$0.01/min (≈4× a turbo render), so audio-embed uploads, kept tracks and ~30 s around reaction ticks, not every track. Stamp vectors with model/dims/chunking.
• Later, owning the model: gisting (compress persona/rules into a few tokens) and xRAG (feed retriever embeddings straight in as tokens) let a local DJ be conditioned on the taste vector directly. Needs an open-weights DJ (Gemma-class) + a trained projector; every kept track is a training pair.

Product

• Market: no shipping product combines per-listener generated songs + a talking DJ + learned taste. Spotify DJ / Yoodio / Trending Music have the voice on licensed catalog; Endel / Noema generate but have no voice; Suno Live Radio etc. are shared broadcasts. The window is cost-driven and won't stay open.
• Pricing sketch (not decided): Free (≈20 new tracks/day per device, then kept/catalog; noise/tones/ASMR unlimited; DJ-read sponsor breaks) · Listener ~$7/mo (unlimited new music, soft cap ~3 h/day generation, no sponsor breaks, all voices, sync) · Studio ~$18/mo (XL, remix/repaint/extend, downloads, own-voice DJ). Price in time and tracks, never credits. Track cost per kept track.
• Uploads / analyzer: upload old tracks → local DSP (bpm, key, energy, sections) + Whisper lyrics + DJ tags → an imported library row → station seeded from it with the file as reference audio.
• Feedback button: 🐛 files a Gitea issue with build rev, player state, current sheet, last ledger lines and console errors attached; dedupe by fingerprint; the factory triages feedback issues.

Rights and moderation

• Runware: outputs owned by the user; model licences are the user's responsibility. Venice: user keeps rights in outputs; API products need end-user terms at least as protective; no third-party voice/likeness without consent. Kokoro: Apache-2.0.
• US Copyright Office: purely AI-generated music isn't copyrightable. Don't promise exclusivity.
• EU AI Act Art. 50 (in force 2 Aug 2026): mark synthetic audio machine-readably (at least ID3 / C2PA-style tag from the worker) and disclose. Applies to EU listeners regardless of where we are.
• To do before anyone outside the team listens: ToS + AI disclosure; output tagging; written confirmation from Runware / ACE Studio on commercial streaming; voice-clone policy (own sample + consent only).
• Moderation: check the sheet (tags, lyrics, DJ line) in the worker before render/TTS; strip real artist and people names from tags; check persona text on save; report button on shared content; per-station explicit setting (hard rules stay on). Venice is permissive, so the guard has to be ours.

POC housekeeping that affects V2

• The module split is done on the split branch (ES modules, no build step: app.js core.js covers.js dj.js render.js store.js player.js mashup.js ui/*.js; index.html is markup + CSS). It is current with main apart from these docs, every module parses, and the Safari harness gives the same results as main. Not yet merged: it needs the dev → phone → main pass. It produces the pure pieces a Swift port reads from (covers math, sheet contract, brain, DJ prompts). docs/v2/REFACTOR-PROMPT.md was written before we saw the branch and is superseded by it.
• tools/harness.sh safari runs the loop in WebKit via safaridriver on hb-5 (needs safaridriver --enable).
• Player changes go dev → phone → main (HANDOFF.md).