site/docs/v2/NOTES-FROM-POC.md

# Notes for V2 from the POC thread (21–22 Oct 2026)

What came out of a long working session on the POC that the V2 spec doesn't already say, or where the POC
experience suggests a different call. Read alongside the V2 handover spec and `HANDOFF.md` (its **Player rules**
section is the iOS lessons; they apply to any client). Nothing here is decided unless it says so.

## Direction

- **Leaning native-native** (Swift/SwiftUI first) over Expo. The reasons that matter for gandora specifically:
  - **The DJ can run on the phone.** iOS 26 Foundation Models does guided generation into a Swift struct; the
    struct is the sheet (`title, tags, lyrics, bpm, key, energy, seconds, meter, line`). Free, offline, ~1 s.
    Rendering still goes to Runware (ACE-Step doesn't fit on a phone).
  - **Testability.** Today's regressions all came from "the only iOS test rig is a phone". XCTest + simulator
    exercise real AVFoundation: background audio, interruptions, lock screen, route changes.
  - `AVQueuePlayer` pre-enqueues the next track, so most of the POC's hold/crackle/`ended` machinery goes away.
- Covers (`hash32`, `prng`, rings/chord/attractor) port line-for-line; keep integer ops identical for parity.
- The web POC stays as the lab, the Android/desktop fallback and the share-link landing. Features are invented
  there and promoted once they've earned it (the spec's parity tracker).
- Alternatives considered: Capacitor (fastest path, wraps the current page), Tauri (early on mobile; good for a
  desktop app with local ACE-Step), Flutter, Expo (right if the UI is being rebuilt anyway).

## Backend

- **Auth: Supabase Web3** (`signInWithWeb3`, Solana/Ethereum, EIP-4361). chai's call.
  - Put the ToS acceptance in `statement` (Venice ToS §7.3 needs end users bound by terms).
  - Wallets are free to mint, so **free tier is per device, not per account**: no login needed to listen;
    login is for sync and paying. Enable per-IP rate limits + captcha.
  - No recovery for wallet-only accounts: nudge `linkIdentity()` once sync is on.
  - iOS PWA + wallets is clunky (deep links / in-wallet browser). Native solves it.
  - No nostr provider; a custom-JWT edge function could add NIP-07 later.
- Minimal first slice: one Supabase project, tables `stations`, `tracks`, `events` with RLS on the wallet
  identity, the existing Cloudflare worker verifying the Supabase JWT, blobs in R2.

## Rendering: ACE-Step and Runware

- ACE-Step 1.5 is **open weights, MIT** (code and weights). Model card: commercial use of outputs allowed;
  training data claimed as licensed + royalty-free/public-domain + synthetic MIDI audio.
- **Runware already exposes repaint and extend** (schema: `runware.ai/docs/models/ace-step-v1-5-xl-turbo/schema.json`):
  - `inputs.audio` (URL/UUID/data URI) + `settings.repaintingStart` / `repaintingEnd` (s) = repaint a section;
    `repaintingEnd` past the end appends (extend), negative `repaintingStart` prepends (intro).
  - `strength` 0–1 with `inputs.audio` = restyle; `numberResults` 1–4 = variations.
  - No stems/extract/lego/cover task types on Runware; those need self-hosting.
  - `duration` must be omitted when `inputs.audio` is set; `timeSignature` is beats (2/3/4/6), not "3/4".
- Product ideas from this: repaint around a reaction pill ("redo this bit"), ⟶ extend, restyle remixes.
- **Self-hosting is for customization, not cost** (Princess's priority). Runware turbo is $0.0045 per 2-min
  track, about what a fully busy GPU costs. Self-hosting buys: LoRAs (per station, per DJ persona, from old
  tracks), the extra task modes, the planner LM, XL without paying double.
  - VRAM tiers (repo): 8–16 GB = 2B turbo + small LM; 20–24 GB = XL turbo without offload.
  - LoRA training: ~1 h for 8 songs on a 12 GB GPU (repo figure).
  - A benchmark of 2B turbo on hb-5 (M1, 8 GB, MLX) was started 21 Oct; result pending at the time of writing.

## Taste: embeddings direction

- One multimodal space (Gemini Embedding 2, 768-d): seed, sheet, uploaded audio, rendered track all as points.
  Station = region (centroid of kept, shadow of avoided); taste = cosine to that direction; far more
  data-efficient than the 54→12→1 net on a few dozen reactions.
- DJ gets **nearest-neighbour exemplars** (RAG for the DJ) instead of `tasteText()`.
- Costs: text embeddings ~free; audio ~$0.01/min (≈4× a turbo render), so audio-embed uploads, kept tracks
  and ~30 s around reaction ticks, not every track. Stamp vectors with model/dims/chunking.
- Later, owning the model: **gisting** (compress persona/rules into a few tokens) and **xRAG** (feed retriever
  embeddings straight in as tokens) let a local DJ be conditioned on the taste vector directly. Needs an
  open-weights DJ (Gemma-class) + a trained projector; every kept track is a training pair.

## Product

- **Market**: no shipping product combines per-listener *generated* songs + a talking DJ + learned taste.
  Spotify DJ / Yoodio / Trending Music have the voice on licensed catalog; Endel / Noema generate but have no
  voice; Suno Live Radio etc. are shared broadcasts. The window is cost-driven and won't stay open.
- **Pricing sketch** (not decided): Free (≈20 new tracks/day per device, then kept/catalog; noise/tones/ASMR
  unlimited; DJ-read sponsor breaks) · Listener ~$7/mo (unlimited new music, soft cap ~3 h/day generation,
  no sponsor breaks, all voices, sync) · Studio ~$18/mo (XL, remix/repaint/extend, downloads, own-voice DJ).
  Price in time and tracks, never credits. Track **cost per kept track**.
- **Uploads / analyzer**: upload old tracks → local DSP (bpm, key, energy, sections) + Whisper lyrics + DJ
  tags → an imported library row → station seeded from it with the file as reference audio.
- **Feedback button**: 🐛 files a Gitea issue with build rev, player state, current sheet, last ledger lines and
  console errors attached; dedupe by fingerprint; the factory triages `feedback` issues.

## Rights and moderation

- Runware: outputs owned by the user; model licences are the user's responsibility. Venice: user keeps rights
  in outputs; API products need end-user terms at least as protective; no third-party voice/likeness without
  consent. Kokoro: Apache-2.0.
- US Copyright Office: purely AI-generated music isn't copyrightable. Don't promise exclusivity.
- **EU AI Act Art. 50** (in force 2 Aug 2026): mark synthetic audio machine-readably (at least ID3 /
  C2PA-style tag from the worker) and disclose. Applies to EU listeners regardless of where we are.
- To do before anyone outside the team listens: ToS + AI disclosure; output tagging; written confirmation from
  Runware / ACE Studio on commercial streaming; voice-clone policy (own sample + consent only).
- **Moderation**: check the sheet (tags, lyrics, DJ line) in the worker before render/TTS; strip real artist and
  people names from tags; check persona text on save; report button on shared content; per-station explicit
  setting (hard rules stay on). Venice is permissive, so the guard has to be ours.

## POC housekeeping that affects V2

- Module split of `index.html` (no build step) is briefed in `docs/v2/REFACTOR-PROMPT.md`. It produces the
  pure pieces a Swift port reads from: covers math, sheet contract, brain, DJ prompts.
- `tools/harness.sh safari` runs the loop in WebKit via safaridriver on hb-5 (needs `safaridriver --enable`).
- Player changes go `dev` → phone → `main` (HANDOFF.md).