Compare commits
84
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
721212ba7e | ||
|
|
08bba21dfa | ||
|
|
7d76ab4338 | ||
|
|
b02e73c0e6 | ||
|
|
d30be0208d | ||
|
|
1da01fcd68 | ||
|
|
0caaad4a1a | ||
|
|
495494b0e7 | ||
|
|
56b74116e7 | ||
|
|
97b7667970 | ||
|
|
195c65cb55 | ||
|
|
f19429bccb | ||
|
|
7e9ae24beb | ||
|
|
c2055e85eb | ||
|
|
b13ddff34f | ||
|
|
f718a14295 | ||
|
|
e46ec071c1 | ||
|
|
10d0466ad9 | ||
|
|
f3a841a762 | ||
|
|
bbc62970cb | ||
|
|
8a14773aa0 | ||
|
|
223a4a0617 | ||
|
|
e12728c5ff | ||
|
|
a694c10428 | ||
|
|
0eb9234a7e | ||
|
|
444b01abca | ||
|
|
4086657dc8 | ||
|
|
aecc4ee90d | ||
|
|
c92313b62d | ||
|
|
dc2a3e8a05 | ||
|
|
a871722024 | ||
|
|
6bf0b0411b | ||
|
|
7c2b6a00a3 | ||
|
|
d05679b11a | ||
|
|
ee0302f763 | ||
|
|
387fe44e15 | ||
|
|
9ffbb793bf | ||
|
|
bd6fcd7ed7 | ||
|
|
8c874e8115 | ||
|
|
c360639456 | ||
|
|
8c6ad054b9 | ||
|
|
17e51a6b08 | ||
|
|
e52e116815 | ||
|
|
2d0f997030 | ||
|
|
3086cb52ff | ||
|
|
9e065e416d | ||
|
|
165d499644 | ||
|
|
8eff78170b | ||
|
|
ba4174326a | ||
|
|
88f9d509ed | ||
|
|
c6451370a7 | ||
|
|
a933c17fdb | ||
|
|
fbe1b0c9a2 | ||
|
|
bc6f232a56 | ||
|
|
c54cbf2596 | ||
|
|
cb687873d3 | ||
|
|
134a03ce41 | ||
|
|
e1dc4f2790 | ||
|
|
32fa6ee49d | ||
|
|
7a821c5cf2 | ||
|
|
6d0c600de0 | ||
|
|
e8843b7e62 | ||
|
|
51811c2ba0 | ||
|
|
99511c6c08 | ||
|
|
398af5a9ac | ||
|
|
c97198ea15 | ||
|
|
23d6383060 | ||
|
|
f64a48300d | ||
|
|
35860f4bf8 | ||
|
|
3bf9fbe94d | ||
|
|
67f7b4d2d9 | ||
|
|
16cd9a6306 | ||
|
|
c287a2acf6 | ||
|
|
4fc82f0f9a | ||
|
|
0f74a7d7c4 | ||
|
|
0c569ca5c2 | ||
|
|
79e38fa894 | ||
|
|
91b11019db | ||
|
|
cdfe4c7199 | ||
|
|
66c427eba4 | ||
|
|
5a185facd3 | ||
|
|
a4970e3e2b | ||
|
|
bae0bdc0ef | ||
|
|
fd23310271 |
No files matched your search
@@ -38,7 +38,7 @@ Files: `.github/workflows/{heartbeat,mention,doctor,update}.yml`, `.github/time/
|
||||
- `gh variable set TIME_MODE --body dry-run`
|
||||
- `gh variable set TIME_DAILY_CAP --body 5` — max agent launches per day
|
||||
- `TIME_WORKER`: `cursor` (default) or `venice:<seat> [model] [effort]`, e.g. `venice:kimi code`. A Venice seat needs only `VENICE_API_KEY`: it reads and writes the repo on branch `time/card-N` through the GitHub API, runs tests on the runner, and opens the PR itself. Workflow files are refused to it.
|
||||
- optional `TIME_CURSOR_MODEL` (Cursor worker model id), `TIME_REVIEW`, `TIME_STUCK_HOURS` (default 6), `TIME_WORKER_STEPS` (default 40 tool calls per card)
|
||||
- optional `TIME_CURSOR_MODEL` (Cursor worker model id), `TIME_REVIEW`, `TIME_STUCK_HOURS` (default 6), `TIME_WORKER_STEPS` (default 40 tool calls per card), `TIME_WORKER_FALLBACK` (default `grok build`: takes over when the worker's model stays overloaded; `off` to disable), `TIME_WORKER_MAX_TOKENS` (default 16000)
|
||||
4. Cursor (the worker, only when `TIME_WORKER` is `cursor`):
|
||||
- The human installs Cursor's GitHub app on this repo (Cursor dashboard → Integrations → GitHub). Cloud agents can't clone or open PRs without it. You can't check this from `gh`; ask them to confirm.
|
||||
- The human creates an API key (Cursor dashboard → API Keys) and stores it: `gh secret set CURSOR_API_KEY` (they paste the value at the prompt).
|
||||
@@ -110,3 +110,20 @@ Only when the human says so: `gh variable set TIME_MODE --body live`. The daily
|
||||
- Runner minutes: private repos have a monthly free allowance, each job rounds up to a minute; keep the heartbeat hourly unless asked. Cursor agent runs bill to the human's Cursor plan.
|
||||
- Changed `time.py`? Run `python3 .github/time/test_time.py` before committing.
|
||||
- Status: `gh issue list -l ready`, `gh run list -L 10`, and the latest Time comment on each card.
|
||||
|
||||
## Ventures (optional)
|
||||
|
||||
`gh variable set TIME_VENTURE --body on` starts the venture loop (`venture.yml`, every 20 minutes): one venture card a day, from the repos trending on GitHub to a launch page on GitHub Pages. Seats pitch, the board votes, board seats write launch plans that one seat synthesizes into `LAUNCH.md`, the worker builds the product as the page on `gh-pages` (the tool fills the first screen), deterministic checks and headless Chrome gate it, and seven days of signups decide (v2, kill, or one more week). It needs `VENICE_API_KEY`; the loop creates the `gh-pages` branch, the Pages site and the `signup` label if it can.
|
||||
- Start one now, whatever the day: `gh workflow run venture.yml -f start=true` (or Actions → venture → Run workflow → "Start a new venture now"). A second card on the same day is titled `Venture <day> (2)`.
|
||||
- Signups come in through the issue form `.github/ISSUE_TEMPLATE/early-access.yml` (label `signup`); the loop thanks each one on its issue. A signup is a distinct GitHub account (not the owner, not a bot) that opened an early-access issue or 👍'd the venture's signal issue.
|
||||
- Settings: `TIME_VENTURE_PITCH`, `TIME_VENTURE_BOARD` (comma-separated seats), `TIME_VENTURE_CHAIR` (synthesizes the launch plans), `TIME_VENTURE_MEASURE_DAYS` (default 7).
|
||||
- A page that still fails its checks after two fix rounds gets `needs-human`: fix it on `gh-pages` and comment `/time retry` on the card. A Venice outage only makes a stage wait; it never closes a card.
|
||||
- The studio index is generated at `/ventures/` from `ventures/ventures.json`; launch posts are drafts on the card, never sent.
|
||||
|
||||
### Backends on Cloudflare (optional)
|
||||
|
||||
With a Cloudflare API token, each venture also gets a backend: its own Worker (`time-<slug>`, code in `.github/time/cloudflare/worker.js`), its own D1 database and Workers AI, serving the venture's Pages folder under its own domain. The page reaches it through `time.js` (written next to `index.html`; the build never edits it): `Time.save` / `Time.load` for share links, `Time.ai` for a small model, and accounts with passkeys (`Time.signUp`, `Time.signIn`, `Time.user`, `Time.mine` for what someone saved). Accounts work only on the venture's own domain: the passkey belongs to that host and the session is a `__Host-` cookie. Every call degrades to null, so a product still works on Pages alone and in the headless checks.
|
||||
- Token: Cloudflare → My Profile → API Tokens → Create Token (custom) with Account · Workers Scripts · Edit, Account · D1 · Edit, Account · Workers AI · Read, and, for a domain, Zone · Workers Routes · Edit and Zone · DNS · Edit on that zone. Then `gh secret set CLOUDFLARE_API_TOKEN` and `gh secret set CLOUDFLARE_ACCOUNT_ID` (the account id is in the dashboard URL).
|
||||
- Domain: `gh variable set TIME_VENTURE_DOMAIN --body example.com` (a zone on that account) puts each venture at `<slug>.example.com`. Without it, ventures live at `time-<slug>.<account>.workers.dev`.
|
||||
- Optional: `TIME_VENTURE_AI_MODEL` (default `@cf/google/gemma-4-26b-a4b-it`).
|
||||
- The doctor checks the token. Existing ventures get backends a couple per tick, and a new `worker.js` (from Time's update) redeploys every venture the same way. A Cloudflare failure never holds a venture: it launches on Pages and the next tick tries again.
|
||||
@@ -0,0 +1,29 @@
|
||||
name: Early access
|
||||
description: Join the early-access list for one of the studio's ventures. No email, nothing charged.
|
||||
title: "Early access: "
|
||||
labels: ["signup"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for raising your hand. This is a **public** GitHub issue: no email is collected and nothing is charged.
|
||||
The studio replies here, and what you write below shapes the next version.
|
||||
- type: input
|
||||
id: venture
|
||||
attributes:
|
||||
label: Venture
|
||||
description: Filled in from the page you came from.
|
||||
placeholder: e.g. mdecks
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: plan
|
||||
attributes:
|
||||
label: Plan
|
||||
description: The tier you would want. Paid tiers are early-access ideas; nothing is charged.
|
||||
placeholder: e.g. Free, or a founding tier
|
||||
- type: textarea
|
||||
id: use
|
||||
attributes:
|
||||
label: What would you use it for?
|
||||
description: One or two sentences about your use. This decides what gets built next.
|
||||
@@ -0,0 +1,157 @@
|
||||
"""Astra: the factory's daily planner. One model call a day that reads the ledger and the board and decides what
|
||||
to work on next. It queues; it never spends on its own beyond its one call.
|
||||
|
||||
python3 .github/time/astra.py tick # daily (astra.yml): read, decide, queue up to a few cards
|
||||
|
||||
Luna is Time's deterministic tick (no model); Sol is the hourly heartbeat that executes cards with a seat; Astra is the
|
||||
slow outer loop that orients them. Astra labels its picks `queued`, never `ready`: the heartbeat's promote_next moves
|
||||
one queued card at a time to `ready` (oldest first, `Depends on #N` respected), so Astra's work lines up behind
|
||||
whatever is already queued instead of running beside it on the same files. Guardrails are code, not prompt: at most
|
||||
TIME_ASTRA_MAX_READY cards queued and TIME_ASTRA_MAX_NEW cards created per run; an existing issue is only queued if it
|
||||
has a `DONE:` line and none of `queued`/`ready`/`needs-human`/`andon`; an open `andon` issue means Astra writes its note
|
||||
and queues nothing; the spend cap is checked by the engine like any other call. Everything Astra does is a comment on
|
||||
one issue, "Astra log", so a human can read the day's reasoning in one place and object there.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
_spec = importlib.util.spec_from_file_location("tm", HERE / "time.py")
|
||||
tm = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(tm)
|
||||
|
||||
SPINE = os.environ.get("TIME_SPINE") or "teahubxyz/teahub" # where AGENTS.md and docs/decisions.md live
|
||||
LOG_TITLE = "Astra log"
|
||||
SKIP = {"queued", "ready", "needs-human", "andon", "astra"} # already moving, a human's, or Astra's own log
|
||||
|
||||
RULES = """You are Astra, the daily planner of a small, fully AI-run software factory (Teahub). Once a day you read the
|
||||
state below and decide what the factory should work on next. You do not do the work: you queue cards, and a worker seat
|
||||
builds queued cards one at a time; you only choose and write cards.
|
||||
|
||||
Rules:
|
||||
- Prefer finishing over starting. Unblock open PRs and half-done cards before opening new work. Cards already labelled
|
||||
`queued` or `ready` are already lined up; don't queue them again.
|
||||
- A card is an issue with a clear list of changes and one `DONE:` line (what must be true for it to be finished, e.g.
|
||||
"DONE: PR open, tests green"). Small, one PR each, a worker can finish in one session.
|
||||
- Never touch workflows, secrets, variables, or anything under docs/decisions.md; those are a human's. Never queue
|
||||
anything that posts outside the repo.
|
||||
- Respect the budget: the day's cap is small. At most {max_ready} cards may be queued and {max_new} created.
|
||||
- If nothing is worth doing, say so; an empty day is fine.
|
||||
|
||||
Answer with JSON only, no prose around it:
|
||||
{{"note": "<3-6 sentences: what the factory did yesterday, what matters today, and why you chose these>",
|
||||
"queue": [<existing open issue numbers to queue now; they must already have a DONE: line>],
|
||||
"new": [{{"title": "<card title>", "body": "<what to change, as a list, ending with a DONE: line>", "queue": true}}]}}"""
|
||||
|
||||
|
||||
def env_int(name: str, default: int) -> int:
|
||||
try:
|
||||
return int(os.environ.get(name) or default)
|
||||
except ValueError:
|
||||
return default
|
||||
|
||||
|
||||
def spine_file(path: str) -> str:
|
||||
try:
|
||||
import base64
|
||||
return base64.b64decode(tm.gh("GET", f"/repos/{SPINE}/contents/{path}")["content"]).decode(errors="replace")
|
||||
except Exception as e: # noqa: BLE001 the spine is context, not a dependency
|
||||
return f"(could not read {SPINE}/{path}: {str(e)[:80]})"
|
||||
|
||||
|
||||
def board() -> dict:
|
||||
"""What Astra reads: open issues and PRs, yesterday's closed issues, andons, spend."""
|
||||
open_issues = tm.issues("state=open")
|
||||
closed = tm.gh("GET", f"/repos/{tm.REPO}/issues?type=issues&state=closed&limit=20")
|
||||
prs = tm.gh("GET", f"/repos/{tm.REPO}/pulls?state=open&limit=20")
|
||||
today, day_cap, month, month_cap = tm.budget()
|
||||
trim = lambda i: {"n": i["number"], "title": i["title"], "labels": [l["name"] for l in i.get("labels") or []],
|
||||
"has_done": bool(re.search(r"^\s*DONE:", i.get("body") or "", re.M | re.I)),
|
||||
"body": (i.get("body") or "")[:600], "updated": i.get("updated_at")}
|
||||
return {"issues": [trim(i) for i in open_issues if i["title"] != LOG_TITLE],
|
||||
"closed_recently": [{"n": i["number"], "title": i["title"], "closed": i.get("closed_at")} for i in closed[:20]],
|
||||
"prs": [{"n": p["number"], "title": p["title"], "head": p["head"]["ref"], "mergeable": p.get("mergeable")} for p in prs],
|
||||
"andons": [i["number"] for i in open_issues if any(l["name"] == "andon" for l in i.get("labels") or [])],
|
||||
"spend": {"today": round(today, 2), "daily_cap": day_cap, "month": round(month, 2), "monthly_cap": month_cap}}
|
||||
|
||||
|
||||
def log_issue() -> int:
|
||||
for i in tm.issues("state=open"):
|
||||
if i["title"] == LOG_TITLE:
|
||||
return i["number"]
|
||||
return tm.new_issue(LOG_TITLE, "Astra, the daily planner, writes one comment here per day: what it read, what it "
|
||||
"queued, and why. Object in a reply; Astra reads this thread too.", ["astra"])["number"]
|
||||
|
||||
|
||||
def decide(state: dict, context: str, seat: str) -> dict:
|
||||
user = f"Factory context (the spine's AGENTS.md and the newest decisions):\n{context}\n\nBoard:\n{json.dumps(state, indent=1)}"
|
||||
raw = tm.think(seat, None, None, RULES.format(max_ready=env_int("TIME_ASTRA_MAX_READY", 2), max_new=env_int("TIME_ASTRA_MAX_NEW", 2)), user)
|
||||
m = re.search(r"\{.*\}", raw, re.S)
|
||||
try:
|
||||
out = json.loads(m.group(0)) if m else {}
|
||||
except ValueError:
|
||||
out = {}
|
||||
picks = out.get("queue") if out.get("queue") is not None else out.get("ready") # older answers said "ready"
|
||||
return {"note": str(out.get("note") or raw[:800]), "queue": [int(n) for n in picks or [] if str(n).isdigit()],
|
||||
"new": [c for c in out.get("new") or [] if isinstance(c, dict) and c.get("title")]}
|
||||
|
||||
|
||||
def tick() -> int:
|
||||
seat = (os.environ.get("TIME_ASTRA_SEAT") or "opus").strip()
|
||||
max_ready, max_new = env_int("TIME_ASTRA_MAX_READY", 2), env_int("TIME_ASTRA_MAX_NEW", 2)
|
||||
state = board()
|
||||
log = log_issue()
|
||||
if state["andons"]:
|
||||
tm.comment(log, f"🔭 Astra: an andon is open (#{', #'.join(map(str, state['andons']))}), so I'm queuing nothing today. "
|
||||
"Close it once it's handled and I'll plan tomorrow.", {"beat": "astra", "queued": 0, "andon": True})
|
||||
return 0
|
||||
context = spine_file("AGENTS.md")[:6000] + "\n\n" + "\n".join(spine_file("docs/decisions.md").splitlines()[-25:])
|
||||
# the log thread is part of the context: humans object there
|
||||
thread = tm.gh("GET", f"/repos/{tm.REPO}/issues/{log}/comments?limit=20")
|
||||
human = [c["body"][:500] for c in thread if not tm.is_time(c)][-5:]
|
||||
if human:
|
||||
context += "\n\nRecent human replies on the Astra log (follow these):\n- " + "\n- ".join(human)
|
||||
try:
|
||||
plan = decide(state, context, seat)
|
||||
except tm.Budget as e:
|
||||
tm.comment(log, f"🔭 Astra: {e}. No plan today.", {"beat": "astra", "queued": 0, "capped": True})
|
||||
return 0
|
||||
by_n = {i["n"]: i for i in state["issues"]}
|
||||
queued, created, skipped = [], [], []
|
||||
for n in plan["queue"][:max_ready]:
|
||||
i = by_n.get(n)
|
||||
if not i or not i["has_done"] or SKIP & set(i["labels"]):
|
||||
skipped.append(n)
|
||||
continue
|
||||
tm.label(n, "queued")
|
||||
queued.append(n)
|
||||
for c in plan["new"][:max_new]:
|
||||
body = str(c.get("body") or "")
|
||||
if not re.search(r"^\s*DONE:", body, re.M | re.I):
|
||||
body += "\n\nDONE: PR open that does the above, tests green."
|
||||
issue = tm.new_issue(str(c["title"])[:200], body + "\n\n<sub>card written by Astra</sub>", [])
|
||||
created.append(issue["number"])
|
||||
if c.get("queue", c.get("ready", True)) and len(queued) + len(created) <= max_ready:
|
||||
tm.label(issue["number"], "queued") # the heartbeat promotes it when nothing else is in flight
|
||||
lines = [f"🔭 Astra ({seat}), spend ${state['spend']['today']:.2f} of ${state['spend']['daily_cap']:.2f} today:", "", plan["note"], ""]
|
||||
if queued:
|
||||
lines.append("Queued: " + ", ".join(f"#{n}" for n in queued))
|
||||
if created:
|
||||
lines.append("New cards: " + ", ".join(f"#{n}" for n in created))
|
||||
if skipped:
|
||||
lines.append("Skipped (no DONE line, already moving, or a human's): " + ", ".join(f"#{n}" for n in skipped))
|
||||
if not (queued or created):
|
||||
lines.append("Queued nothing today.")
|
||||
tm.comment(log, "\n".join(lines), {"beat": "astra", "queued": len(queued) + len(created), "queue": queued, "new": created})
|
||||
print(f"astra: queued={queued} new={created} skipped={skipped}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(tick() if sys.argv[1:2] == ["tick"] else __doc__)
|
||||
@@ -0,0 +1,177 @@
|
||||
"""Cloudflare backing for ventures: each venture gets a domain, a Worker, a D1 database and Workers AI.
|
||||
|
||||
python3 .github/time/cloudflare.py check # what the token can do (doctor)
|
||||
python3 .github/time/cloudflare.py deploy <slug> <pages-folder-url>
|
||||
|
||||
Off unless CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID are set. The page stays on GitHub Pages; the Worker
|
||||
(cloudflare/worker.js, one script `time-<slug>` per venture) serves it under the venture's own domain and adds the API
|
||||
the page's time.js client calls: share links stored in D1, and a rate-limited small model. With TIME_VENTURE_DOMAIN set
|
||||
(a zone on the same Cloudflare account) a venture lives at <slug>.<domain>; without it, at the script's workers.dev URL.
|
||||
|
||||
Everything here is idempotent: deploying again updates the script in place and reuses the database and the domain.
|
||||
The deployed revision (REV) is a hash of the worker code and its settings, so the venture loop can redeploy every
|
||||
venture when Time's update brings a new worker, and leave them alone otherwise.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
API = "https://api.cloudflare.com/client/v4"
|
||||
HERE = Path(__file__).resolve().parent
|
||||
WORKER = HERE / "cloudflare" / "worker.js"
|
||||
CLIENT = HERE / "cloudflare" / "time.js"
|
||||
COMPAT = "2026-09-01"
|
||||
AI_MODEL = "@cf/google/gemma-4-26b-a4b-it"
|
||||
|
||||
|
||||
def env(name: str, default: str = "") -> str:
|
||||
return (os.environ.get(name) or default).strip()
|
||||
|
||||
|
||||
def enabled() -> bool:
|
||||
return bool(env("CLOUDFLARE_API_TOKEN") and env("CLOUDFLARE_ACCOUNT_ID"))
|
||||
|
||||
|
||||
def account() -> str:
|
||||
return f"/accounts/{env('CLOUDFLARE_ACCOUNT_ID')}"
|
||||
|
||||
|
||||
def cf(method: str, path: str, body=None, *, data: bytes | None = None, ctype: str = "application/json"):
|
||||
"""One Cloudflare API call; returns `result`. Errors name the call and Cloudflare's own messages, never the token."""
|
||||
raw = data if data is not None else (json.dumps(body).encode() if body is not None else None)
|
||||
req = urllib.request.Request(API + path, method=method, data=raw,
|
||||
headers={"Authorization": f"Bearer {env('CLOUDFLARE_API_TOKEN')}", "Content-Type": ctype})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=60) as r:
|
||||
out = json.loads(r.read() or b"{}")
|
||||
except urllib.error.HTTPError as e:
|
||||
text = e.read().decode(errors="replace")
|
||||
try:
|
||||
msgs = "; ".join(f"{x.get('code')} {x.get('message')}" for x in json.loads(text).get("errors", []))
|
||||
except ValueError:
|
||||
msgs = text[:200]
|
||||
raise RuntimeError(f"cloudflare {method} {path.split('?')[0].replace(env('CLOUDFLARE_ACCOUNT_ID'), '<account>')} "
|
||||
f"-> {e.code}: {msgs}") from None
|
||||
return out.get("result")
|
||||
|
||||
|
||||
def script_name(slug: str) -> str:
|
||||
return re.sub(r"[^a-z0-9-]", "-", f"time-{slug}".lower())[:63].strip("-")
|
||||
|
||||
|
||||
def host(slug: str) -> str | None:
|
||||
zone = env("TIME_VENTURE_DOMAIN").lower().strip(".")
|
||||
return f"{slug}.{zone}" if zone else None
|
||||
|
||||
|
||||
def settings(slug: str, origin: str, pages_origin: str) -> dict:
|
||||
return {"SLUG": slug, "ORIGIN": origin.rstrip("/"), "PAGES_ORIGIN": pages_origin,
|
||||
"AI_MODEL": env("TIME_VENTURE_AI_MODEL", AI_MODEL), "AI_DAILY": env("TIME_VENTURE_AI_DAILY", "200"),
|
||||
"SAVE_DAILY": env("TIME_VENTURE_SAVE_DAILY", "2000")}
|
||||
|
||||
|
||||
def rev(vars_: dict) -> str:
|
||||
"""The revision a deploy would make: worker code + settings + domain. Equal revs mean nothing to do."""
|
||||
h = hashlib.sha256(WORKER.read_bytes())
|
||||
h.update(json.dumps({**vars_, "host": host(vars_["SLUG"]), "compat": COMPAT}, sort_keys=True).encode())
|
||||
return h.hexdigest()[:12]
|
||||
|
||||
|
||||
def database(slug: str) -> str:
|
||||
"""The venture's own D1 database (created once). Returns its id."""
|
||||
name = script_name(slug)
|
||||
for d in cf("GET", f"{account()}/d1/database?name={name}") or []:
|
||||
if d.get("name") == name:
|
||||
return d["uuid"]
|
||||
return cf("POST", f"{account()}/d1/database", {"name": name})["uuid"]
|
||||
|
||||
|
||||
def multipart(parts: list[tuple[str, str, bytes]]) -> tuple[bytes, str]:
|
||||
b = uuid.uuid4().hex
|
||||
out = b"".join(f'--{b}\r\nContent-Disposition: form-data; name="{n}"; filename="{n}"\r\nContent-Type: {t}\r\n\r\n'.encode()
|
||||
+ data + b"\r\n" for n, t, data in parts) + f"--{b}--\r\n".encode()
|
||||
return out, f"multipart/form-data; boundary={b}"
|
||||
|
||||
|
||||
def upload(name: str, db: str, vars_: dict) -> None:
|
||||
meta = {"main_module": "worker.js", "compatibility_date": COMPAT,
|
||||
"bindings": [{"type": "d1", "name": "DB", "id": db}, {"type": "ai", "name": "AI"}]
|
||||
+ [{"type": "plain_text", "name": k, "text": str(v)} for k, v in vars_.items()]}
|
||||
data, ctype = multipart([("metadata", "application/json", json.dumps(meta).encode()),
|
||||
("worker.js", "application/javascript+module", WORKER.read_bytes())])
|
||||
cf("PUT", f"{account()}/workers/scripts/{name}", data=data, ctype=ctype)
|
||||
|
||||
|
||||
def attach(name: str, hostname: str) -> str:
|
||||
"""Point <slug>.<domain> at the script as a Workers custom domain (Cloudflare adds the DNS record and the cert)."""
|
||||
zone = hostname.split(".", 1)[1]
|
||||
zones = cf("GET", f"/zones?name={zone}") or []
|
||||
if not zones:
|
||||
raise RuntimeError(f"TIME_VENTURE_DOMAIN {zone} is not a zone this Cloudflare token can see")
|
||||
cf("PUT", f"{account()}/workers/domains",
|
||||
{"hostname": hostname, "service": name, "zone_id": zones[0]["id"], "environment": "production"})
|
||||
return f"https://{hostname}/"
|
||||
|
||||
|
||||
def workers_dev(name: str) -> str:
|
||||
cf("POST", f"{account()}/workers/scripts/{name}/subdomain", {"enabled": True, "previews_enabled": False})
|
||||
sub = (cf("GET", f"{account()}/workers/subdomain") or {}).get("subdomain")
|
||||
if not sub:
|
||||
raise RuntimeError("this Cloudflare account has no workers.dev subdomain yet: set TIME_VENTURE_DOMAIN instead")
|
||||
return f"https://{name}.{sub}.workers.dev/"
|
||||
|
||||
|
||||
def pages_origin_of(url: str) -> str:
|
||||
m = re.match(r"(https?://[^/]+)", url)
|
||||
return m.group(1) if m else ""
|
||||
|
||||
|
||||
def deploy(slug: str, origin: str) -> dict:
|
||||
"""Database, script and address for one venture. Returns {url, worker, db, rev, api}."""
|
||||
vars_ = settings(slug, origin, pages_origin_of(origin))
|
||||
vars_["REV"] = r = rev(vars_)
|
||||
name = script_name(slug)
|
||||
db = database(slug)
|
||||
upload(name, db, vars_)
|
||||
hostname = host(slug)
|
||||
url = attach(name, hostname) if hostname else workers_dev(name)
|
||||
return {"url": url, "worker": name, "db": db, "rev": r, "api": url + "api/"}
|
||||
|
||||
|
||||
def client(api: str) -> str:
|
||||
"""The time.js a venture page loads, pointed at its backend."""
|
||||
return CLIENT.read_text().replace("__TIME_API__", api)
|
||||
|
||||
|
||||
def check() -> tuple[bool, str]:
|
||||
"""Can this token do what a venture needs? For the doctor: (ok, detail), with nothing secret in the detail."""
|
||||
if not enabled():
|
||||
return False, "not set (optional): CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID give ventures domains, a database and AI"
|
||||
try:
|
||||
cf("GET", f"{account()}/workers/scripts")
|
||||
cf("GET", f"{account()}/d1/database?per_page=1")
|
||||
zone = env("TIME_VENTURE_DOMAIN")
|
||||
if zone and not cf("GET", f"/zones?name={zone}"):
|
||||
return False, f"works, but TIME_VENTURE_DOMAIN {zone} is not a zone this token can see"
|
||||
return True, f"works: Workers and D1; ventures live at {'<slug>.' + zone if zone else 'workers.dev'}"
|
||||
except Exception as e: # noqa: BLE001
|
||||
return False, f"set but failing: {str(e)[:160]}"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if sys.argv[1:2] == ["check"]:
|
||||
ok, detail = check()
|
||||
print(detail)
|
||||
sys.exit(0 if ok else 1)
|
||||
if sys.argv[1:2] == ["deploy"] and len(sys.argv) == 4:
|
||||
print(json.dumps(deploy(sys.argv[2], sys.argv[3]), indent=1))
|
||||
sys.exit(0)
|
||||
sys.exit(__doc__)
|
||||
@@ -0,0 +1,187 @@
|
||||
// Offline test for passkey accounts in worker.js: a real SQLite standing in for D1 (node:sqlite) and a software
|
||||
// authenticator with real P-256 and RSA keys. Run: node .github/time/cloudflare/test_auth.mjs
|
||||
// Skips (and says so) when this node has no node:sqlite.
|
||||
import assert from "node:assert/strict";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
let DatabaseSync;
|
||||
try {
|
||||
process.removeAllListeners("warning");
|
||||
({ DatabaseSync } = await import("node:sqlite"));
|
||||
} catch {
|
||||
console.log("AUTH SKIPPED: no node:sqlite");
|
||||
process.exit(0);
|
||||
}
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const worker = (await import(join(here, "worker.js"))).default;
|
||||
|
||||
// D1's surface over SQLite: prepare/bind/run/first/all, and batch as one transaction.
|
||||
class D1 {
|
||||
constructor() { this.db = new DatabaseSync(":memory:"); }
|
||||
prepare(sql) {
|
||||
const db = this.db;
|
||||
const make = (args) => ({
|
||||
bind: (...a) => make(a),
|
||||
exec: () => ({ results: db.prepare(sql).all(...args) }),
|
||||
run: async () => ({ results: db.prepare(sql).all(...args) }),
|
||||
first: async () => db.prepare(sql).all(...args)[0] ?? null,
|
||||
all: async () => ({ results: db.prepare(sql).all(...args) }),
|
||||
});
|
||||
return make([]);
|
||||
}
|
||||
async batch(stmts) {
|
||||
this.db.exec("BEGIN");
|
||||
try {
|
||||
const out = stmts.map((s) => s.exec());
|
||||
this.db.exec("COMMIT");
|
||||
return out;
|
||||
} catch (e) {
|
||||
this.db.exec("ROLLBACK");
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const HOST = "mdecks.yolo.lol", ORIGIN = `https://${HOST}`;
|
||||
const env = { DB: new D1(), SLUG: "mdecks", REV: "r", ORIGIN: "https://o.github.io/r/ventures/mdecks/", PAGES_ORIGIN: "https://o.github.io" };
|
||||
const enc = new TextEncoder();
|
||||
const b64u = (buf) => Buffer.from(buf).toString("base64url");
|
||||
const unb64u = (s) => new Uint8Array(Buffer.from(s, "base64url"));
|
||||
const sha256 = async (d) => new Uint8Array(await crypto.subtle.digest("SHA-256", typeof d === "string" ? enc.encode(d) : d));
|
||||
let ip = 0;
|
||||
const call = (path, { body, cookie, origin, host = HOST } = {}) => worker.fetch(new Request(`https://${host}${path}`, {
|
||||
method: body === undefined ? "GET" : "POST", body: body === undefined ? undefined : JSON.stringify(body),
|
||||
headers: { "content-type": "application/json", "cf-connecting-ip": `10.0.0.${++ip % 250}`, ...(cookie ? { cookie } : {}),
|
||||
...(origin ? { origin } : {}) } }), env);
|
||||
const cookieOf = (r) => (r.headers.get("set-cookie") || "").split(";")[0];
|
||||
|
||||
// The software authenticator: what a phone or a laptop's passkey would hand the page.
|
||||
function authData(rpHash, flags, counter) {
|
||||
const a = new Uint8Array(37);
|
||||
a.set(rpHash);
|
||||
a[32] = flags;
|
||||
new DataView(a.buffer).setUint32(33, counter);
|
||||
return a;
|
||||
}
|
||||
function rawToDer(raw) {
|
||||
const int = (b) => {
|
||||
let v = [...b];
|
||||
while (v.length > 1 && v[0] === 0 && !(v[1] & 0x80)) v.shift();
|
||||
if (v[0] & 0x80) v = [0, ...v];
|
||||
return [0x02, v.length, ...v];
|
||||
};
|
||||
const body = [...int(raw.slice(0, 32)), ...int(raw.slice(32))];
|
||||
return new Uint8Array([0x30, body.length, ...body]);
|
||||
}
|
||||
class Authenticator {
|
||||
static async make(kind = "ES256") {
|
||||
const a = new Authenticator();
|
||||
a.kind = kind;
|
||||
a.alg = kind === "ES256" ? -7 : -257;
|
||||
a.keys = kind === "ES256"
|
||||
? await crypto.subtle.generateKey({ name: "ECDSA", namedCurve: "P-256" }, true, ["sign", "verify"])
|
||||
: await crypto.subtle.generateKey({ name: "RSASSA-PKCS1-v1_5", modulusLength: 2048, publicExponent: new Uint8Array([1, 0, 1]),
|
||||
hash: "SHA-256" }, true, ["sign", "verify"]);
|
||||
a.id = b64u(crypto.getRandomValues(new Uint8Array(16)));
|
||||
a.counter = 0;
|
||||
return a;
|
||||
}
|
||||
async create(options, { origin = ORIGIN } = {}) {
|
||||
const clientData = enc.encode(JSON.stringify({ type: "webauthn.create", challenge: options.publicKey.challenge, origin }));
|
||||
return { challengeId: options.challengeId, id: this.id, alg: this.alg, clientDataJSON: b64u(clientData),
|
||||
authenticatorData: b64u(authData(await sha256(options.publicKey.rp.id), 0x45, this.counter)),
|
||||
publicKey: b64u(await crypto.subtle.exportKey("spki", this.keys.publicKey)) };
|
||||
}
|
||||
async get(options, { origin = ORIGIN, counter = ++this.counter, rpId = options.publicKey.rpId } = {}) {
|
||||
const clientData = enc.encode(JSON.stringify({ type: "webauthn.get", challenge: options.publicKey.challenge, origin }));
|
||||
const ad = authData(await sha256(rpId), 0x05, counter);
|
||||
const signed = new Uint8Array([...ad, ...(await sha256(clientData))]);
|
||||
const algo = this.kind === "ES256" ? { name: "ECDSA", hash: "SHA-256" } : { name: "RSASSA-PKCS1-v1_5" };
|
||||
let sig = new Uint8Array(await crypto.subtle.sign(algo, this.keys.privateKey, signed));
|
||||
if (this.kind === "ES256") sig = rawToDer(sig);
|
||||
return { challengeId: options.challengeId, id: this.id, clientDataJSON: b64u(clientData), authenticatorData: b64u(ad),
|
||||
signature: b64u(sig) };
|
||||
}
|
||||
}
|
||||
|
||||
// sign up: options for this host, then the passkey's answer makes the account and a session
|
||||
const key = await Authenticator.make();
|
||||
let r = await call("/api/auth/register/start", { body: { name: "Ada" } });
|
||||
let opts = await r.json();
|
||||
assert.equal(opts.publicKey.rp.id, HOST);
|
||||
assert.equal(opts.publicKey.user.name, "Ada");
|
||||
assert.deepEqual(opts.publicKey.pubKeyCredParams.map((p) => p.alg), [-7, -257]);
|
||||
r = await call("/api/auth/register/finish", { body: await key.create(opts) });
|
||||
assert.equal(r.status, 200, await r.clone().text());
|
||||
const ada = (await r.json()).user;
|
||||
assert.equal(ada.name, "Ada");
|
||||
let cookie = cookieOf(r);
|
||||
assert.match(r.headers.get("set-cookie"), /^__Host-time=[A-Za-z0-9_-]+; Path=\/; Secure; HttpOnly; SameSite=Lax; Max-Age=2592000$/);
|
||||
|
||||
// the session knows who it is, and keeps what they save
|
||||
assert.deepEqual((await (await call("/api/auth/me", { cookie })).json()).user, ada);
|
||||
r = await call("/api/items", { body: { deck: "# mine" }, cookie });
|
||||
const mine = (await r.json()).id;
|
||||
await call("/api/items", { body: { deck: "# someone else's" } });
|
||||
r = await call("/api/me/items", { cookie });
|
||||
assert.deepEqual((await r.json()).items.map((i) => i.id), [mine]);
|
||||
assert.equal((await call("/api/me/items")).status, 401);
|
||||
|
||||
// sign out, then sign back in with the same passkey (discoverable: no username asked)
|
||||
r = await call("/api/auth/logout", { body: {}, cookie });
|
||||
assert.match(r.headers.get("set-cookie"), /Max-Age=0/);
|
||||
assert.equal((await call("/api/auth/me", { cookie })).status, 401);
|
||||
opts = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
assert.equal(opts.publicKey.rpId, HOST);
|
||||
const answer = await key.get(opts);
|
||||
r = await call("/api/auth/login/finish", { body: answer });
|
||||
assert.equal(r.status, 200, await r.clone().text());
|
||||
cookie = cookieOf(r);
|
||||
assert.deepEqual((await (await call("/api/auth/me", { cookie })).json()).user, ada);
|
||||
assert.deepEqual((await (await call("/api/me/items", { cookie })).json()).items.map((i) => i.id), [mine]);
|
||||
|
||||
// what must fail
|
||||
const fails = async (path, body, why) => {
|
||||
const res = await call(path, { body });
|
||||
assert.equal(res.status, 400, `${why}: ${res.status}`);
|
||||
return (await res.json()).error;
|
||||
};
|
||||
assert.match(await fails("/api/auth/login/finish", answer, "replayed answer"), /expired/);
|
||||
let o = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
assert.match(await fails("/api/auth/login/finish", await key.get(o, { origin: "https://evil.example" }), "phishing origin"), /origin/);
|
||||
o = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
assert.match(await fails("/api/auth/login/finish", await key.get(o, { rpId: "evil.example" }), "other site's passkey"), /relying party/);
|
||||
o = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
assert.match(await fails("/api/auth/login/finish", await key.get(o, { counter: 1 }), "counter went back"), /cloned/);
|
||||
o = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
const forged = await key.get(o);
|
||||
const bad = unb64u(forged.signature);
|
||||
bad[bad.length - 1] ^= 1;
|
||||
assert.match(await fails("/api/auth/login/finish", { ...forged, signature: b64u(bad) }, "tampered signature"), /signature/);
|
||||
o = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
const stranger = await Authenticator.make();
|
||||
assert.match(await fails("/api/auth/login/finish", await stranger.get(o), "unregistered passkey"), /unknown passkey/);
|
||||
o = await (await call("/api/auth/register/start", { body: { name: "Again" } })).json();
|
||||
assert.match(await fails("/api/auth/register/finish", await key.create(o), "same passkey twice"), /already registered/);
|
||||
o = await (await call("/api/auth/register/start", { body: { name: "Eve" } })).json();
|
||||
assert.match(await fails("/api/auth/register/finish", await stranger.create(o, { origin: "https://o.github.io" }), "from Pages"), /origin/);
|
||||
assert.equal((await call("/api/auth/login/start", { body: {}, origin: "https://o.github.io" })).status, 403);
|
||||
assert.equal((await call("/api/auth/me", { cookie: "__Host-time=forged" })).status, 401);
|
||||
// a failed sign-up left no half-made account behind
|
||||
assert.equal(env.DB.db.prepare("SELECT COUNT(*) AS n FROM users").all()[0].n, 1);
|
||||
|
||||
// RSA passkeys (RS256, some Windows Hello setups) work too
|
||||
const rsa = await Authenticator.make("RS256");
|
||||
o = await (await call("/api/auth/register/start", { body: { name: "Grace" } })).json();
|
||||
r = await call("/api/auth/register/finish", { body: await rsa.create(o) });
|
||||
assert.equal(r.status, 200, await r.clone().text());
|
||||
o = await (await call("/api/auth/login/start", { body: {} })).json();
|
||||
r = await call("/api/auth/login/finish", { body: await rsa.get(o) });
|
||||
assert.equal((await r.json()).user.name, "Grace");
|
||||
|
||||
// the other endpoints still work on a database that has accounts
|
||||
r = await call("/api/health");
|
||||
assert.equal((await r.json()).ok, true);
|
||||
|
||||
console.log("AUTH OK");
|
||||
@@ -0,0 +1,113 @@
|
||||
// Offline test for worker.js: a fake D1 (just the statements the worker uses), fake Workers AI and a fake Pages origin.
|
||||
// Run: node .github/time/cloudflare/test_worker.mjs (test_time.py runs it when node is around)
|
||||
import assert from "node:assert/strict";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const worker = (await import(join(here, "worker.js"))).default;
|
||||
|
||||
class FakeD1 {
|
||||
constructor() { this.items = new Map(); this.quota = new Map(); this.created = 0; }
|
||||
prepare(sql) {
|
||||
const db = this;
|
||||
const stmt = {
|
||||
args: [],
|
||||
bind(...a) { return { ...stmt, args: a }; },
|
||||
async run() { return this.exec(); },
|
||||
async first() { return (await this.exec()).results[0] ?? null; },
|
||||
async exec() {
|
||||
const a = this.args;
|
||||
if (sql.startsWith("CREATE TABLE")) { db.created++; return { results: [] }; }
|
||||
if (sql.startsWith("INSERT INTO quota")) {
|
||||
const n = (db.quota.get(a[0]) || 0) + 1; db.quota.set(a[0], n); return { results: [{ n }] };
|
||||
}
|
||||
if (sql.startsWith("INSERT INTO items")) { db.items.set(a[0], a[1]); return { results: [] }; }
|
||||
if (sql.startsWith("SELECT data FROM items")) { return { results: db.items.has(a[0]) ? [{ data: db.items.get(a[0]) }] : [] }; }
|
||||
throw new Error("unexpected sql " + sql);
|
||||
},
|
||||
};
|
||||
return stmt;
|
||||
}
|
||||
async batch(stmts) { return Promise.all(stmts.map((s) => s.exec())); }
|
||||
}
|
||||
|
||||
const prompts = [];
|
||||
const env = {
|
||||
DB: new FakeD1(), SLUG: "mdecks", REV: "abc123", ORIGIN: "https://o.github.io/r/ventures/mdecks/", PAGES_ORIGIN: "https://o.github.io",
|
||||
AI_MODEL: "@cf/test/model", AI_DAILY: "3", SAVE_DAILY: "100",
|
||||
AI: { async run(model, input) { prompts.push({ model, input }); return { response: "a crisp title" }; } },
|
||||
};
|
||||
const pages = [];
|
||||
globalThis.fetch = async (url, init) => {
|
||||
pages.push({ url, init });
|
||||
if (url.endsWith("/ventures/mdecks/shots")) return new Response(null, { status: 301, headers: { location: "https://o.github.io/r/ventures/mdecks/shots/" } });
|
||||
return new Response(`<html>${url}</html>`, { status: 200, headers: { "content-type": "text/html" } });
|
||||
};
|
||||
const call = (path, init = {}, ip = "1.1.1.1") =>
|
||||
worker.fetch(new Request("https://mdecks.yolo.lol" + path, { ...init, headers: { "cf-connecting-ip": ip, ...(init.headers || {}) } }), env);
|
||||
const post = (path, body, ip, headers = {}) =>
|
||||
call(path, { method: "POST", body: typeof body === "string" ? body : JSON.stringify(body), headers: { "content-type": "application/json", ...headers } }, ip);
|
||||
|
||||
// health, without touching the database
|
||||
let r = await call("/api/health");
|
||||
assert.deepEqual(await r.json(), { ok: true, slug: "mdecks", rev: "abc123", ai: true });
|
||||
assert.equal(env.DB.created, 0);
|
||||
|
||||
// save and load a creation; the link comes back to this domain
|
||||
r = await post("/api/items", { slides: ["# Hi"] });
|
||||
assert.equal(r.status, 201);
|
||||
const saved = await r.json();
|
||||
assert.match(saved.id, /^[a-f0-9]{12}$/);
|
||||
assert.equal(saved.link, `https://mdecks.yolo.lol/?s=${saved.id}`);
|
||||
assert.equal(env.DB.created, 7); // every table, created once
|
||||
r = await call(`/api/items/${saved.id}`);
|
||||
assert.deepEqual(await r.json(), { slides: ["# Hi"] });
|
||||
assert.equal((await call("/api/items/doesnotexist1")).status, 404);
|
||||
assert.equal((await post("/api/items", "not json")).status, 400);
|
||||
assert.equal((await post("/api/items", JSON.stringify({ x: "y".repeat(70000) }))).status, 413);
|
||||
|
||||
// AI through Workers AI with the configured model; per-person and daily limits
|
||||
r = await post("/api/ai", { prompt: "title for my deck", system: "be brief" }, "2.2.2.2");
|
||||
assert.deepEqual(await r.json(), { text: "a crisp title" });
|
||||
assert.equal(prompts[0].model, "@cf/test/model");
|
||||
assert.deepEqual(prompts[0].input.messages.map((m) => m.role), ["system", "user"]);
|
||||
assert.equal((await post("/api/ai", { prompt: "" }, "2.2.2.2")).status, 400);
|
||||
assert.equal((await post("/api/ai", { prompt: "two" }, "3.3.3.3")).status, 200);
|
||||
assert.equal((await post("/api/ai", { prompt: "three" }, "4.4.4.4")).status, 200);
|
||||
assert.equal((await post("/api/ai", { prompt: "over the day" }, "5.5.5.5")).status, 429);
|
||||
assert.equal(prompts.length, 3);
|
||||
|
||||
// newer models answer in the OpenAI shape
|
||||
const openai = { ...env, DB: new FakeD1(), AI: { async run() { return { choices: [{ message: { content: "from choices" } }] }; } } };
|
||||
r = await worker.fetch(new Request("https://mdecks.yolo.lol/api/ai", { method: "POST", body: JSON.stringify({ prompt: "hi" }),
|
||||
headers: { "cf-connecting-ip": "9.9.9.9" } }), openai);
|
||||
assert.deepEqual(await r.json(), { text: "from choices" });
|
||||
|
||||
// CORS: this domain and the Pages origin, nobody else
|
||||
r = await call("/api/health", { headers: { origin: "https://o.github.io" } });
|
||||
assert.equal(r.headers.get("access-control-allow-origin"), "https://o.github.io");
|
||||
r = await call("/api/health", { headers: { origin: "https://evil.example" } });
|
||||
assert.equal(r.headers.get("access-control-allow-origin"), null);
|
||||
assert.equal((await call("/api/nope")).status, 404);
|
||||
|
||||
// a broken database answers 500 with JSON, not an exception
|
||||
const broken = { ...env, DB: { prepare() { throw new Error("D1 down"); }, batch() { return Promise.reject(new Error("D1 down")); } } };
|
||||
r = await worker.fetch(new Request("https://x.yolo.lol/api/items", { method: "POST", body: "{}" }), broken);
|
||||
assert.equal(r.status, 500);
|
||||
assert.match((await r.json()).detail, /D1 down/);
|
||||
|
||||
// the page: Pages under this domain, query kept, redirects kept on this host
|
||||
r = await call("/?s=" + saved.id);
|
||||
assert.equal(pages.at(-1).url, "https://o.github.io/r/ventures/mdecks/?s=" + saved.id);
|
||||
assert.equal(r.headers.get("x-time-venture"), "mdecks");
|
||||
assert.match(await r.text(), /ventures\/mdecks\//);
|
||||
r = await call("/app.js");
|
||||
assert.equal(pages.at(-1).url, "https://o.github.io/r/ventures/mdecks/app.js");
|
||||
r = await call("");
|
||||
assert.equal(pages.at(-1).url, "https://o.github.io/r/ventures/mdecks/");
|
||||
r = await call("/shots");
|
||||
assert.equal(r.status, 301);
|
||||
assert.equal(r.headers.get("location"), "/shots/");
|
||||
|
||||
console.log("WORKER OK");
|
||||
@@ -0,0 +1,94 @@
|
||||
/* Time's backend client for a venture page. Written by Time next to index.html; the build never edits it.
|
||||
Every call resolves to null (or false) when the backend can't help (no network, over the daily limit, or the page is
|
||||
being checked locally), so a product uses these only for extras and works fully without them.
|
||||
Time.save(obj) -> a share link (string) or null; when someone is signed in it is also theirs
|
||||
Time.load() -> the object a share link (?s=...) carried, or null
|
||||
Time.ai(prompt, {system}) -> text from a small model, or null
|
||||
Time.ready -> a promise of true when the backend answers
|
||||
Accounts, with passkeys (Face ID, Touch ID, Windows Hello, a phone): no passwords, no email.
|
||||
Time.accounts -> true when this page can sign people in (its own domain, a browser with passkeys)
|
||||
Time.user() -> { id, name } of whoever is signed in, or null
|
||||
Time.signUp(name) -> makes an account with a new passkey and signs in: { id, name } or null
|
||||
Time.signIn() -> signs in with a passkey made here before: { id, name } or null
|
||||
Time.signOut() -> true
|
||||
Time.mine() -> [{ id, link, created }] the signed-in person saved, or null */
|
||||
(function () {
|
||||
var API = "__TIME_API__";
|
||||
var local = /^(localhost|127\.|\[::1\]|0\.0\.0\.0)/.test(location.hostname) || location.protocol === "file:";
|
||||
var home = API.replace(/\/api\/?$/, "");
|
||||
var accounts = !local && location.origin === home.replace(/\/$/, "") && !!window.PublicKeyCredential;
|
||||
|
||||
function call(path, init) {
|
||||
if (local) return Promise.resolve(null);
|
||||
return fetch(API + path, init).then(function (r) { return r.ok ? r.json() : null; }).catch(function () { return null; });
|
||||
}
|
||||
function post(path, body) {
|
||||
return call(path, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body || {}) });
|
||||
}
|
||||
function bytes(s) {
|
||||
s = s.replace(/-/g, "+").replace(/_/g, "/");
|
||||
return Uint8Array.from(atob(s + "===".slice((s.length + 3) % 4)), function (c) { return c.charCodeAt(0); });
|
||||
}
|
||||
function text(buf) {
|
||||
var b = new Uint8Array(buf), s = "";
|
||||
for (var i = 0; i < b.length; i++) s += String.fromCharCode(b[i]);
|
||||
return btoa(s).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
|
||||
}
|
||||
function user(r) { return r && r.user ? r.user : null; }
|
||||
|
||||
window.Time = {
|
||||
api: API,
|
||||
accounts: accounts,
|
||||
ready: call("health").then(function (r) { return !!(r && r.ok); }),
|
||||
save: function (obj) {
|
||||
return post("items", obj).then(function (r) { return r && r.link ? r.link : null; });
|
||||
},
|
||||
load: function () {
|
||||
var id = new URLSearchParams(location.search).get("s");
|
||||
return id && /^[a-z0-9]{6,32}$/.test(id) ? call("items/" + id) : Promise.resolve(null);
|
||||
},
|
||||
ai: function (prompt, opts) {
|
||||
return post("ai", { prompt: String(prompt || ""), system: opts && opts.system })
|
||||
.then(function (r) { return r && typeof r.text === "string" ? r.text : null; });
|
||||
},
|
||||
user: function () {
|
||||
return accounts ? call("auth/me").then(user) : Promise.resolve(null);
|
||||
},
|
||||
signUp: function (name) {
|
||||
if (!accounts) return Promise.resolve(null);
|
||||
return post("auth/register/start", { name: name }).then(function (o) {
|
||||
if (!o) return null;
|
||||
var pk = o.publicKey;
|
||||
pk.challenge = bytes(pk.challenge);
|
||||
pk.user.id = bytes(pk.user.id);
|
||||
return navigator.credentials.create({ publicKey: pk }).then(function (c) {
|
||||
var r = c.response;
|
||||
return post("auth/register/finish", {
|
||||
challengeId: o.challengeId, id: text(c.rawId), clientDataJSON: text(r.clientDataJSON),
|
||||
authenticatorData: text(r.getAuthenticatorData()), publicKey: text(r.getPublicKey()), alg: r.getPublicKeyAlgorithm()
|
||||
}).then(user);
|
||||
});
|
||||
}).catch(function () { return null; });
|
||||
},
|
||||
signIn: function () {
|
||||
if (!accounts) return Promise.resolve(null);
|
||||
return post("auth/login/start").then(function (o) {
|
||||
if (!o) return null;
|
||||
o.publicKey.challenge = bytes(o.publicKey.challenge);
|
||||
return navigator.credentials.get({ publicKey: o.publicKey }).then(function (c) {
|
||||
var r = c.response;
|
||||
return post("auth/login/finish", {
|
||||
challengeId: o.challengeId, id: text(c.rawId), clientDataJSON: text(r.clientDataJSON),
|
||||
authenticatorData: text(r.authenticatorData), signature: text(r.signature)
|
||||
}).then(user);
|
||||
});
|
||||
}).catch(function () { return null; });
|
||||
},
|
||||
signOut: function () {
|
||||
return accounts ? post("auth/logout").then(function () { return true; }) : Promise.resolve(true);
|
||||
},
|
||||
mine: function () {
|
||||
return accounts ? call("me/items").then(function (r) { return r && r.items ? r.items : null; }) : Promise.resolve(null);
|
||||
}
|
||||
};
|
||||
})();
|
||||
@@ -0,0 +1,308 @@
|
||||
// A venture's backend on Cloudflare: its own domain in front of its GitHub Pages folder, plus a small API.
|
||||
// Time deploys one copy per venture (script time-<slug>) with its own D1 database; see .github/time/cloudflare.py.
|
||||
//
|
||||
// GET /api/health -> { ok, slug, rev, ai }
|
||||
// POST /api/items -> { id, link } save a JSON creation (<= 64 KB), for share links
|
||||
// GET /api/items/:id -> the saved JSON
|
||||
// POST /api/ai -> { text } { prompt, system? } through Workers AI, rate limited
|
||||
// accounts, with passkeys (WebAuthn): no passwords, no email, nothing to configure
|
||||
// POST /api/auth/register/start { name } -> { challengeId, publicKey } options for navigator.credentials.create
|
||||
// POST /api/auth/register/finish -> { user } sets the session cookie
|
||||
// POST /api/auth/login/start -> { challengeId, publicKey } options for navigator.credentials.get
|
||||
// POST /api/auth/login/finish -> { user } sets the session cookie
|
||||
// GET /api/auth/me -> { user } or 401 POST /api/auth/logout
|
||||
// GET /api/me/items -> { items } what the signed-in person saved (POST /api/items records the owner)
|
||||
// anything else -> the venture's page on GitHub Pages (ORIGIN), served under this domain
|
||||
//
|
||||
// Accounts live on the venture's own domain: the passkey's relying party is this host, and the session is a
|
||||
// __Host- cookie (HttpOnly, Secure, SameSite=Lax). On GitHub Pages or in the checks there is no session, by design.
|
||||
//
|
||||
// Bindings: DB (D1), AI (Workers AI). Vars: SLUG, ORIGIN, PAGES_ORIGIN, REV, AI_MODEL, AI_DAILY, SAVE_DAILY.
|
||||
|
||||
const SCHEMA = [
|
||||
"CREATE TABLE IF NOT EXISTS items (id TEXT PRIMARY KEY, data TEXT NOT NULL, created TEXT NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS quota (k TEXT PRIMARY KEY, n INTEGER NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS users (id TEXT PRIMARY KEY, name TEXT NOT NULL, created TEXT NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS creds (id TEXT PRIMARY KEY, user_id TEXT NOT NULL, spki TEXT NOT NULL, alg INTEGER NOT NULL, " +
|
||||
"counter INTEGER NOT NULL, created TEXT NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS sessions (hash TEXT PRIMARY KEY, user_id TEXT NOT NULL, expires INTEGER NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS challenges (id TEXT PRIMARY KEY, challenge TEXT NOT NULL, kind TEXT NOT NULL, user_id TEXT, " +
|
||||
"name TEXT, expires INTEGER NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS owned (item_id TEXT PRIMARY KEY, user_id TEXT NOT NULL, created TEXT NOT NULL)",
|
||||
];
|
||||
const COOKIE = "__Host-time";
|
||||
const SESSION_DAYS = 30;
|
||||
const CHALLENGE_MS = 5 * 60 * 1000;
|
||||
const MAX_ITEM = 65536;
|
||||
const MAX_PROMPT = 4000;
|
||||
let ready = null;
|
||||
|
||||
const json = (body, status = 200, headers = {}) =>
|
||||
new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json; charset=utf-8", ...headers } });
|
||||
|
||||
function cors(req, env) {
|
||||
const origin = req.headers.get("origin");
|
||||
if (!origin) return {};
|
||||
const self = new URL(req.url).origin;
|
||||
const ok = origin === self || (env.PAGES_ORIGIN && origin === env.PAGES_ORIGIN);
|
||||
return ok ? { "access-control-allow-origin": origin, "access-control-allow-methods": "GET, POST",
|
||||
"access-control-allow-headers": "content-type", "vary": "origin" } : {};
|
||||
}
|
||||
|
||||
async function setup(env) {
|
||||
ready ??= env.DB.batch(SCHEMA.map((s) => env.DB.prepare(s))).catch((e) => { ready = null; throw e; });
|
||||
return ready;
|
||||
}
|
||||
|
||||
async function who(req, env) {
|
||||
const ip = req.headers.get("cf-connecting-ip") || "?";
|
||||
const raw = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(`${env.SLUG}:${ip}`));
|
||||
return [...new Uint8Array(raw).slice(0, 8)].map((b) => b.toString(16).padStart(2, "0")).join("");
|
||||
}
|
||||
|
||||
// One counter per (day, kind, person) and one per (day, kind): over either limit means no.
|
||||
async function spend(env, kind, person, perPerson, total) {
|
||||
const day = new Date().toISOString().slice(0, 10);
|
||||
const bump = (k) => env.DB.prepare("INSERT INTO quota (k, n) VALUES (?, 1) ON CONFLICT(k) DO UPDATE SET n = n + 1 RETURNING n").bind(k);
|
||||
const [mine, all] = await env.DB.batch([bump(`${day}:${kind}:${person}`), bump(`${day}:${kind}:*`)]);
|
||||
return mine.results[0].n <= perPerson && all.results[0].n <= total;
|
||||
}
|
||||
|
||||
// ---------- passkeys (WebAuthn) ----------
|
||||
|
||||
const enc = new TextEncoder();
|
||||
const b64u = (buf) => btoa(String.fromCharCode(...new Uint8Array(buf))).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
|
||||
const unb64u = (s) => Uint8Array.from(atob(String(s).replace(/-/g, "+").replace(/_/g, "/")), (c) => c.charCodeAt(0));
|
||||
const random = (n) => crypto.getRandomValues(new Uint8Array(n));
|
||||
const sha256 = async (data) => new Uint8Array(await crypto.subtle.digest("SHA-256", typeof data === "string" ? enc.encode(data) : data));
|
||||
const same = (a, b) => a.length === b.length && a.every((x, i) => x === b[i]);
|
||||
const ALGS = {
|
||||
"-7": { importAs: { name: "ECDSA", namedCurve: "P-256" }, verifyAs: { name: "ECDSA", hash: "SHA-256" }, der: true },
|
||||
"-257": { importAs: { name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" }, verifyAs: { name: "RSASSA-PKCS1-v1_5" }, der: false },
|
||||
};
|
||||
|
||||
// Authenticators sign ES256 as DER (SEQUENCE of two INTEGERs); WebCrypto wants r || s, 32 bytes each.
|
||||
function derToRaw(der) {
|
||||
let i = 2;
|
||||
if (der[1] & 0x80) i += der[1] & 0x7f;
|
||||
const part = () => {
|
||||
if (der[i++] !== 0x02) throw new Error("bad signature");
|
||||
const len = der[i++];
|
||||
let v = der.slice(i, i + len);
|
||||
i += len;
|
||||
while (v.length > 32 && v[0] === 0) v = v.slice(1);
|
||||
const out = new Uint8Array(32);
|
||||
out.set(v, 32 - v.length);
|
||||
return out;
|
||||
};
|
||||
const r = part(), s = part();
|
||||
const raw = new Uint8Array(64);
|
||||
raw.set(r);
|
||||
raw.set(s, 32);
|
||||
return raw;
|
||||
}
|
||||
|
||||
async function challenge(env, kind, extra = {}) {
|
||||
const id = b64u(random(12)), value = b64u(random(32)), now = Date.now();
|
||||
await env.DB.batch([
|
||||
env.DB.prepare("DELETE FROM challenges WHERE expires < ?").bind(now),
|
||||
env.DB.prepare("INSERT INTO challenges (id, challenge, kind, user_id, name, expires) VALUES (?, ?, ?, ?, ?, ?)")
|
||||
.bind(id, value, kind, extra.user ?? null, extra.name ?? null, now + CHALLENGE_MS),
|
||||
]);
|
||||
return { id, value };
|
||||
}
|
||||
|
||||
// One use only: the row is gone whether or not the answer checks out.
|
||||
async function takeChallenge(env, id, kind) {
|
||||
const row = await env.DB.prepare("DELETE FROM challenges WHERE id = ? RETURNING challenge, kind, user_id, name, expires")
|
||||
.bind(String(id || "")).first();
|
||||
if (!row || row.kind !== kind || row.expires < Date.now()) throw new AuthError("that sign-in attempt expired; try again");
|
||||
return row;
|
||||
}
|
||||
|
||||
class AuthError extends Error {}
|
||||
|
||||
// What the browser and the authenticator both vouched for: the ceremony, our challenge, our origin, our host.
|
||||
async function checkCeremony(url, type, expected, clientDataB64, authDataB64) {
|
||||
const clientRaw = unb64u(clientDataB64);
|
||||
const client = JSON.parse(new TextDecoder().decode(clientRaw));
|
||||
if (client.type !== type) throw new AuthError("wrong ceremony");
|
||||
if (client.challenge !== expected) throw new AuthError("challenge mismatch");
|
||||
if (client.origin !== url.origin) throw new AuthError("wrong origin");
|
||||
const auth = unb64u(authDataB64);
|
||||
if (auth.length < 37 || !same(auth.slice(0, 32), await sha256(url.hostname))) throw new AuthError("wrong relying party");
|
||||
const flags = auth[32];
|
||||
if (!(flags & 0x01)) throw new AuthError("no user presence");
|
||||
const counter = new DataView(auth.buffer, auth.byteOffset + 33, 4).getUint32(0);
|
||||
return { clientRaw, auth, flags, counter };
|
||||
}
|
||||
|
||||
async function startSession(env, url, userId, h) {
|
||||
const token = b64u(random(32));
|
||||
const expires = Date.now() + SESSION_DAYS * 86400e3;
|
||||
await env.DB.prepare("INSERT INTO sessions (hash, user_id, expires) VALUES (?, ?, ?)").bind(b64u(await sha256(token)), userId, expires).run();
|
||||
const user = await env.DB.prepare("SELECT id, name FROM users WHERE id = ?").bind(userId).first();
|
||||
return json({ user }, 200, { ...h, "set-cookie": `${COOKIE}=${token}; Path=/; Secure; HttpOnly; SameSite=Lax; Max-Age=${SESSION_DAYS * 86400}` });
|
||||
}
|
||||
|
||||
function cookieToken(req) {
|
||||
const m = (req.headers.get("cookie") || "").match(new RegExp(`(?:^|;\\s*)${COOKIE}=([A-Za-z0-9_-]+)`));
|
||||
return m ? m[1] : null;
|
||||
}
|
||||
|
||||
async function currentUser(req, env) {
|
||||
const token = cookieToken(req);
|
||||
if (!token) return null;
|
||||
return env.DB.prepare("SELECT users.id, users.name FROM sessions JOIN users ON users.id = sessions.user_id " +
|
||||
"WHERE sessions.hash = ? AND sessions.expires > ?").bind(b64u(await sha256(token)), Date.now()).first();
|
||||
}
|
||||
|
||||
async function auth(req, env, url, path, h) {
|
||||
// Accounts belong to this domain: refuse cross-origin calls outright.
|
||||
const origin = req.headers.get("origin");
|
||||
if (origin && origin !== url.origin) return json({ error: "sign in on the venture's own domain" }, 403, h);
|
||||
if (path === "/api/auth/me") {
|
||||
const user = await currentUser(req, env);
|
||||
return user ? json({ user }, 200, h) : json({ error: "not signed in" }, 401, h);
|
||||
}
|
||||
if (path === "/api/auth/logout" && req.method === "POST") {
|
||||
const token = cookieToken(req);
|
||||
if (token) await env.DB.prepare("DELETE FROM sessions WHERE hash = ?").bind(b64u(await sha256(token))).run();
|
||||
return json({ ok: true }, 200, { ...h, "set-cookie": `${COOKIE}=; Path=/; Secure; HttpOnly; SameSite=Lax; Max-Age=0` });
|
||||
}
|
||||
if (req.method !== "POST") return json({ error: "no such endpoint" }, 404, h);
|
||||
let body;
|
||||
try { body = await req.json(); } catch { return json({ error: "send JSON" }, 400, h); }
|
||||
try {
|
||||
if (path === "/api/auth/register/start") {
|
||||
if (!(await spend(env, "register", await who(req, env), 20, 2000))) return json({ error: "over today's limit" }, 429, h);
|
||||
const name = String(body.name || "").trim().slice(0, 40) || "Me";
|
||||
const user = b64u(random(16));
|
||||
const c = await challenge(env, "register", { user, name });
|
||||
return json({ challengeId: c.id, publicKey: {
|
||||
challenge: c.value, rp: { name: env.SLUG, id: url.hostname }, user: { id: user, name, displayName: name },
|
||||
pubKeyCredParams: [{ type: "public-key", alg: -7 }, { type: "public-key", alg: -257 }],
|
||||
authenticatorSelection: { residentKey: "required", userVerification: "preferred" }, attestation: "none", timeout: 60000,
|
||||
} }, 200, h);
|
||||
}
|
||||
if (path === "/api/auth/register/finish") {
|
||||
const c = await takeChallenge(env, body.challengeId, "register");
|
||||
const { flags, counter } = await checkCeremony(url, "webauthn.create", c.challenge, body.clientDataJSON, body.authenticatorData);
|
||||
if (!(flags & 0x40)) throw new AuthError("no credential in the answer");
|
||||
const alg = ALGS[String(body.alg)];
|
||||
if (!alg || !body.publicKey || !/^[A-Za-z0-9_-]{16,1400}$/.test(String(body.id))) throw new AuthError("unsupported passkey");
|
||||
await crypto.subtle.importKey("spki", unb64u(body.publicKey), alg.importAs, false, ["verify"]); // throws if it isn't a key
|
||||
const now = new Date().toISOString();
|
||||
try {
|
||||
await env.DB.batch([
|
||||
env.DB.prepare("INSERT INTO users (id, name, created) VALUES (?, ?, ?)").bind(c.user_id, c.name, now),
|
||||
env.DB.prepare("INSERT INTO creds (id, user_id, spki, alg, counter, created) VALUES (?, ?, ?, ?, ?, ?)")
|
||||
.bind(body.id, c.user_id, body.publicKey, Number(body.alg), counter, now),
|
||||
]);
|
||||
} catch { throw new AuthError("that passkey is already registered here"); }
|
||||
return startSession(env, url, c.user_id, h);
|
||||
}
|
||||
if (path === "/api/auth/login/start") {
|
||||
if (!(await spend(env, "login", await who(req, env), 60, 5000))) return json({ error: "over today's limit" }, 429, h);
|
||||
const c = await challenge(env, "login");
|
||||
return json({ challengeId: c.id, publicKey: { challenge: c.value, rpId: url.hostname, userVerification: "preferred",
|
||||
timeout: 60000 } }, 200, h);
|
||||
}
|
||||
if (path === "/api/auth/login/finish") {
|
||||
const c = await takeChallenge(env, body.challengeId, "login");
|
||||
const { clientRaw, auth: authData, counter } = await checkCeremony(url, "webauthn.get", c.challenge, body.clientDataJSON,
|
||||
body.authenticatorData);
|
||||
const cred = await env.DB.prepare("SELECT user_id, spki, alg, counter FROM creds WHERE id = ?").bind(String(body.id || "")).first();
|
||||
if (!cred) throw new AuthError("unknown passkey for this site");
|
||||
const alg = ALGS[String(cred.alg)];
|
||||
const key = await crypto.subtle.importKey("spki", unb64u(cred.spki), alg.importAs, false, ["verify"]);
|
||||
const signed = new Uint8Array(authData.length + 32);
|
||||
signed.set(authData);
|
||||
signed.set(await sha256(clientRaw), authData.length);
|
||||
let sig = unb64u(body.signature);
|
||||
if (alg.der) sig = derToRaw(sig);
|
||||
if (!(await crypto.subtle.verify(alg.verifyAs, key, sig, signed))) throw new AuthError("signature does not check out");
|
||||
if ((counter || cred.counter) && counter <= cred.counter) throw new AuthError("this passkey looks cloned");
|
||||
await env.DB.prepare("UPDATE creds SET counter = ? WHERE id = ?").bind(counter, body.id).run();
|
||||
return startSession(env, url, cred.user_id, h);
|
||||
}
|
||||
} catch (e) {
|
||||
if (e instanceof AuthError) return json({ error: e.message }, 400, h);
|
||||
throw e;
|
||||
}
|
||||
return json({ error: "no such endpoint" }, 404, h);
|
||||
}
|
||||
|
||||
async function api(req, env, url, h) {
|
||||
const path = url.pathname.replace(/\/+$/, "");
|
||||
if (req.method === "OPTIONS") return new Response(null, { status: 204, headers: h });
|
||||
if (path === "/api/health") return json({ ok: true, slug: env.SLUG, rev: env.REV, ai: !!env.AI }, 200, h);
|
||||
await setup(env);
|
||||
if (path.startsWith("/api/auth/")) return auth(req, env, url, path, h);
|
||||
if (path === "/api/me/items") {
|
||||
const user = await currentUser(req, env);
|
||||
if (!user) return json({ error: "not signed in" }, 401, h);
|
||||
const rows = await env.DB.prepare("SELECT item_id, created FROM owned WHERE user_id = ? ORDER BY created DESC LIMIT 50")
|
||||
.bind(user.id).all();
|
||||
return json({ items: rows.results.map((r) => ({ id: r.item_id, link: `${url.origin}/?s=${r.item_id}`, created: r.created })) }, 200, h);
|
||||
}
|
||||
if (path === "/api/items" && req.method === "POST") {
|
||||
const text = await req.text();
|
||||
if (text.length > MAX_ITEM) return json({ error: "too big (64 KB max)" }, 413, h);
|
||||
try { JSON.parse(text); } catch { return json({ error: "send JSON" }, 400, h); }
|
||||
if (!(await spend(env, "save", await who(req, env), 50, Number(env.SAVE_DAILY || 2000))))
|
||||
return json({ error: "over today's limit" }, 429, h);
|
||||
const id = crypto.randomUUID().replace(/-/g, "").slice(0, 12);
|
||||
const now = new Date().toISOString();
|
||||
const user = await currentUser(req, env);
|
||||
await env.DB.batch([env.DB.prepare("INSERT INTO items (id, data, created) VALUES (?, ?, ?)").bind(id, text, now),
|
||||
...(user ? [env.DB.prepare("INSERT INTO owned (item_id, user_id, created) VALUES (?, ?, ?)").bind(id, user.id, now)] : [])]);
|
||||
return json({ id, link: `${url.origin}/?s=${id}` }, 201, h);
|
||||
}
|
||||
const item = path.match(/^\/api\/items\/([a-z0-9]{6,32})$/);
|
||||
if (item && req.method === "GET") {
|
||||
const row = await env.DB.prepare("SELECT data FROM items WHERE id = ?").bind(item[1]).first();
|
||||
return row ? new Response(row.data, { headers: { "content-type": "application/json; charset=utf-8", ...h } })
|
||||
: json({ error: "not found" }, 404, h);
|
||||
}
|
||||
if (path === "/api/ai" && req.method === "POST") {
|
||||
if (!env.AI) return json({ error: "no AI on this venture" }, 501, h);
|
||||
let body;
|
||||
try { body = await req.json(); } catch { return json({ error: "send JSON" }, 400, h); }
|
||||
const prompt = String(body.prompt || "").slice(0, MAX_PROMPT);
|
||||
if (!prompt) return json({ error: "prompt is empty" }, 400, h);
|
||||
if (!(await spend(env, "ai", await who(req, env), 20, Number(env.AI_DAILY || 200))))
|
||||
return json({ error: "over today's limit" }, 429, h);
|
||||
const messages = [...(body.system ? [{ role: "system", content: String(body.system).slice(0, MAX_PROMPT) }] : []),
|
||||
{ role: "user", content: prompt }];
|
||||
const out = await env.AI.run(env.AI_MODEL, { messages, max_tokens: 512 });
|
||||
// older models answer { response }, newer ones the OpenAI shape { choices: [{ message: { content } }] }
|
||||
const text = out.response ?? out.choices?.[0]?.message?.content ?? "";
|
||||
return json({ text: typeof text === "string" ? text : JSON.stringify(text) }, 200, h);
|
||||
}
|
||||
return json({ error: "no such endpoint" }, 404, h);
|
||||
}
|
||||
|
||||
// The page itself: the venture's folder on GitHub Pages, under this domain. Pages redirects folder URLs without a
|
||||
// trailing slash to its own host; those are rewritten to stay here.
|
||||
async function page(req, env, url) {
|
||||
const origin = env.ORIGIN.replace(/\/+$/, "");
|
||||
const up = await fetch(origin + url.pathname + url.search, { redirect: "manual", cf: { cacheTtl: 60, cacheEverything: true } });
|
||||
const headers = new Headers(up.headers);
|
||||
const loc = headers.get("location");
|
||||
if (loc && loc.startsWith(origin)) headers.set("location", loc.slice(origin.length) || "/");
|
||||
headers.set("x-time-venture", env.SLUG);
|
||||
return new Response(up.body, { status: up.status, headers });
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(req, env) {
|
||||
const url = new URL(req.url);
|
||||
if (url.pathname === "/api" || url.pathname.startsWith("/api/")) {
|
||||
const h = cors(req, env);
|
||||
try { return await api(req, env, url, h); }
|
||||
catch (e) { return json({ error: "backend error", detail: String(e).slice(0, 200) }, 500, h); }
|
||||
}
|
||||
return page(req, env, url);
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,235 @@
|
||||
// AICharts: natural-language charts for the Chart.js you already have, without sending your rows to a model.
|
||||
//
|
||||
// The model sees a dataset's schema, a few sample rows and aggregates, never the data itself. It answers with a small
|
||||
// JSON chart spec. The spec is validated, then a fixed set of operations (filter, group, sort, bin, limit) computes the
|
||||
// chart's data in the browser. No model-written code runs anywhere.
|
||||
//
|
||||
// attach(chart, { data, schema, endpoint }) -> { ask(prompt) } drive an existing Chart.js chart
|
||||
// <ai-chart endpoint="..." dataset="..."> a framework-free element: prompt box, chart, save
|
||||
// validateSpec(spec, schema) -> { ok, errors, fields }
|
||||
// applySpec(rows, spec) -> rows the fixed operations
|
||||
// chartConfig(rows, spec) -> a Chart.js config
|
||||
//
|
||||
// A spec:
|
||||
// { "title": "...", "mark": "bar" | "line" | "pie" | "doughnut" | "scatter",
|
||||
// "transform": [ { "op": "filter", "field": "region", "in": ["West", "East"] },
|
||||
// { "op": "group", "by": ["month"], "agg": "sum", "field": "amount", "as": "value" },
|
||||
// { "op": "sort", "field": "month", "order": "asc" }, { "op": "limit", "n": 12 } ],
|
||||
// "x": "month", "y": "value", "series": "region" (optional) }
|
||||
|
||||
export const MARKS = ["bar", "line", "pie", "doughnut", "scatter"];
|
||||
export const AGGS = ["sum", "avg", "count", "min", "max"];
|
||||
export const OPS = ["filter", "group", "sort", "bin", "limit"];
|
||||
const MAX_STEPS = 8;
|
||||
const NAME = /^[A-Za-z_][A-Za-z0-9_]{0,40}$/;
|
||||
|
||||
// schema: { columns: [{ name, type: "string" | "number" | "month" | "date" }] }
|
||||
export function validateSpec(spec, schema) {
|
||||
const errors = [];
|
||||
const err = (m) => { errors.push(m); return { ok: false, errors }; };
|
||||
if (!spec || typeof spec !== "object" || Array.isArray(spec)) return err("the spec must be a JSON object");
|
||||
if (!MARKS.includes(spec.mark)) err(`mark must be one of ${MARKS.join(", ")}`);
|
||||
const types = new Map((schema?.columns || []).map((c) => [c.name, c.type]));
|
||||
const steps = spec.transform ?? [];
|
||||
if (!Array.isArray(steps)) return err("transform must be a list of steps");
|
||||
if (steps.length > MAX_STEPS) err(`at most ${MAX_STEPS} transform steps`);
|
||||
const has = (f) => types.has(f);
|
||||
steps.forEach((s, i) => {
|
||||
const at = `step ${i + 1}`;
|
||||
if (!s || typeof s !== "object" || !OPS.includes(s.op)) { err(`${at}: op must be one of ${OPS.join(", ")}`); return; }
|
||||
if (s.op === "filter") {
|
||||
if (!has(s.field)) { err(`${at}: unknown field ${JSON.stringify(s.field)}`); return; }
|
||||
const keys = ["eq", "in", "gte", "lte"].filter((k) => k in s);
|
||||
if (!keys.length) err(`${at}: a filter needs eq, in, gte or lte`);
|
||||
if ("in" in s && (!Array.isArray(s.in) || s.in.length > 50)) err(`${at}: in must be a list of at most 50 values`);
|
||||
} else if (s.op === "group") {
|
||||
const by = s.by ?? [];
|
||||
if (!Array.isArray(by) || by.length > 2 || !by.every(has)) { err(`${at}: by must list up to 2 known fields`); return; }
|
||||
if (!AGGS.includes(s.agg)) { err(`${at}: agg must be one of ${AGGS.join(", ")}`); return; }
|
||||
if (s.agg !== "count" && types.get(s.field) !== "number") { err(`${at}: ${s.agg} needs a number field`); return; }
|
||||
const as = s.as ?? "value";
|
||||
if (!NAME.test(as)) { err(`${at}: bad output name ${JSON.stringify(as)}`); return; }
|
||||
const next = new Map(by.map((f) => [f, types.get(f)]));
|
||||
next.set(as, "number");
|
||||
types.clear();
|
||||
next.forEach((t, f) => types.set(f, t));
|
||||
} else if (s.op === "sort") {
|
||||
if (!has(s.field)) err(`${at}: unknown field ${JSON.stringify(s.field)}`);
|
||||
if (s.order && !["asc", "desc"].includes(s.order)) err(`${at}: order must be asc or desc`);
|
||||
} else if (s.op === "bin") {
|
||||
if (types.get(s.field) !== "number") { err(`${at}: bin needs a number field`); return; }
|
||||
if (!(Number(s.step) > 0)) err(`${at}: step must be a positive number`);
|
||||
const as = s.as ?? `${s.field}_bin`;
|
||||
if (!NAME.test(as)) { err(`${at}: bad output name`); return; }
|
||||
types.set(as, "number");
|
||||
} else if (s.op === "limit") {
|
||||
if (!Number.isInteger(s.n) || s.n < 1 || s.n > 500) err(`${at}: n must be a whole number from 1 to 500`);
|
||||
}
|
||||
});
|
||||
for (const k of ["x", "y"]) if (!has(spec[k])) err(`${k} must be a field the transform leaves (got ${JSON.stringify(spec[k])})`);
|
||||
if (has(spec.y) && types.get(spec.y) !== "number") err("y must be a number field");
|
||||
if (spec.series != null && !has(spec.series)) err(`series must be a field the transform leaves`);
|
||||
if (spec.mark === "scatter" && types.get(spec.x) !== "number") err("a scatter needs a number on x");
|
||||
if (spec.title != null && typeof spec.title !== "string") err("title must be text");
|
||||
return { ok: errors.length === 0, errors, fields: Object.fromEntries(types) };
|
||||
}
|
||||
|
||||
const cmp = (a, b) => (a < b ? -1 : a > b ? 1 : 0);
|
||||
|
||||
export function applySpec(rows, spec) {
|
||||
let out = rows.map((r) => ({ ...r }));
|
||||
for (const s of spec.transform ?? []) {
|
||||
if (s.op === "filter") {
|
||||
out = out.filter((r) => {
|
||||
const v = r[s.field];
|
||||
if ("eq" in s && v != s.eq) return false; // loose on purpose: "2026" matches 2026
|
||||
if ("in" in s && !s.in.some((x) => x == v)) return false;
|
||||
if ("gte" in s && !(v >= s.gte)) return false;
|
||||
if ("lte" in s && !(v <= s.lte)) return false;
|
||||
return true;
|
||||
});
|
||||
} else if (s.op === "group") {
|
||||
const by = s.by ?? [], as = s.as ?? "value", groups = new Map();
|
||||
for (const r of out) {
|
||||
const key = JSON.stringify(by.map((f) => r[f]));
|
||||
if (!groups.has(key)) groups.set(key, { row: Object.fromEntries(by.map((f) => [f, r[f]])), vals: [] });
|
||||
groups.get(key).vals.push(s.agg === "count" ? 1 : Number(r[s.field]));
|
||||
}
|
||||
out = [...groups.values()].map(({ row, vals }) => {
|
||||
const sum = vals.reduce((a, b) => a + b, 0);
|
||||
const v = { sum, count: vals.length, avg: sum / (vals.length || 1), min: Math.min(...vals), max: Math.max(...vals) }[s.agg];
|
||||
return { ...row, [as]: Math.round(v * 100) / 100 };
|
||||
});
|
||||
} else if (s.op === "sort") {
|
||||
const dir = s.order === "desc" ? -1 : 1;
|
||||
out = [...out].sort((a, b) => dir * cmp(a[s.field], b[s.field]));
|
||||
} else if (s.op === "bin") {
|
||||
const as = s.as ?? `${s.field}_bin`, step = Number(s.step);
|
||||
out = out.map((r) => ({ ...r, [as]: Math.floor(Number(r[s.field]) / step) * step }));
|
||||
} else if (s.op === "limit") {
|
||||
out = out.slice(0, s.n);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const PALETTE = ["#2f6fdf", "#e0752d", "#2f9e6e", "#c2417a", "#7a5bd6", "#9a8a2a", "#3aa0b8", "#b8573a"];
|
||||
|
||||
export function chartConfig(rows, spec) {
|
||||
const title = spec.title ? { display: true, text: spec.title } : { display: false };
|
||||
if (spec.mark === "scatter") {
|
||||
return { type: "scatter", data: { datasets: [{ label: spec.y, data: rows.map((r) => ({ x: Number(r[spec.x]), y: Number(r[spec.y]) })),
|
||||
backgroundColor: PALETTE[0] }] }, options: { plugins: { title }, animation: false } };
|
||||
}
|
||||
const labels = [...new Set(rows.map((r) => r[spec.x]))];
|
||||
let datasets;
|
||||
if (spec.series && !["pie", "doughnut"].includes(spec.mark)) {
|
||||
const names = [...new Set(rows.map((r) => r[spec.series]))];
|
||||
datasets = names.map((name, i) => ({
|
||||
label: String(name),
|
||||
data: labels.map((l) => rows.find((r) => r[spec.x] === l && r[spec.series] === name)?.[spec.y] ?? null),
|
||||
backgroundColor: PALETTE[i % PALETTE.length], borderColor: PALETTE[i % PALETTE.length],
|
||||
}));
|
||||
} else {
|
||||
const round = ["pie", "doughnut"].includes(spec.mark);
|
||||
datasets = [{ label: spec.y, data: labels.map((l) => rows.find((r) => r[spec.x] === l)?.[spec.y] ?? null),
|
||||
backgroundColor: round ? labels.map((_, i) => PALETTE[i % PALETTE.length]) : PALETTE[0], borderColor: PALETTE[0] }];
|
||||
}
|
||||
return { type: spec.mark, data: { labels: labels.map(String), datasets },
|
||||
options: { plugins: { title, legend: { display: datasets.length > 1 || ["pie", "doughnut"].includes(spec.mark) } }, animation: false } };
|
||||
}
|
||||
|
||||
async function fetchSpec(endpoint, body) {
|
||||
const r = await fetch(endpoint.replace(/\/$/, "") + "/api/spec", { method: "POST", headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify(body) });
|
||||
const out = await r.json().catch(() => ({}));
|
||||
if (!r.ok) throw new Error(out.error || `the chart service answered ${r.status}`);
|
||||
return out; // { spec, brain?, model?, ms?, confidence?, alternatives?, handoff? }
|
||||
}
|
||||
|
||||
// Drive an existing Chart.js chart. `data` stays in the browser; only `schema` (plus a sample and aggregates the server
|
||||
// computes for its own datasets) shapes the prompt.
|
||||
export function attach(chart, { data, schema, endpoint, dataset }) {
|
||||
let last;
|
||||
return {
|
||||
async ask(prompt, prior = last, brain) {
|
||||
const { spec } = await fetchSpec(endpoint, { dataset, prompt, prior, ...(brain ? { brain } : {}) });
|
||||
const v = validateSpec(spec, schema);
|
||||
if (!v.ok) throw new Error("the spec failed validation: " + v.errors.join("; "));
|
||||
const cfg = chartConfig(applySpec(data, spec), spec);
|
||||
chart.config.type = cfg.type;
|
||||
chart.config.data = cfg.data;
|
||||
chart.config.options = { ...chart.config.options, ...cfg.options };
|
||||
chart.update();
|
||||
last = spec;
|
||||
return spec;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const Base = globalThis.HTMLElement ?? class {};
|
||||
|
||||
// <ai-chart endpoint="https://..." dataset="orders" brain="jev"></ai-chart> brain: llm | jev | clef (server default if unset)
|
||||
// Needs Chart.js on the page (window.Chart). Fires "spec" after each answer and "save" when Save is pressed; set
|
||||
// element.onSave = async ({ spec, prompt, dataset }) => ... to store charts yourself (the demo stores them in D1).
|
||||
export class AiChart extends Base {
|
||||
connectedCallback() {
|
||||
if (this._ready) return;
|
||||
this._ready = true;
|
||||
this.innerHTML = `<form class="aic-ask"><input name="q" placeholder="Ask for a chart, e.g. sales by region" aria-label="Chart question"
|
||||
autocomplete="off"><button>Chart it</button></form><div class="aic-msg" role="status"></div>
|
||||
<div class="aic-canvas"><canvas></canvas></div><button class="aic-save" hidden>Save chart</button>`;
|
||||
this._msg = this.querySelector(".aic-msg");
|
||||
this.querySelector("form").addEventListener("submit", (e) => { e.preventDefault(); this.ask(e.target.q.value); });
|
||||
this.querySelector(".aic-save").addEventListener("click", () => this.save());
|
||||
}
|
||||
|
||||
get endpoint() { return this.getAttribute("endpoint") || location.origin; }
|
||||
get dataset() { return this.getAttribute("dataset") || "orders"; }
|
||||
|
||||
async load() {
|
||||
if (this._data) return;
|
||||
const base = this.endpoint.replace(/\/$/, "");
|
||||
const [schema, data] = await Promise.all([fetch(`${base}/api/schema?dataset=${this.dataset}`).then((r) => r.json()),
|
||||
fetch(`${base}/api/data?dataset=${this.dataset}`).then((r) => r.json())]);
|
||||
this._schema = schema.schema;
|
||||
this._data = data.rows;
|
||||
}
|
||||
|
||||
async ask(prompt) {
|
||||
if (!prompt?.trim()) return;
|
||||
this._msg.textContent = "Thinking…";
|
||||
try {
|
||||
await this.load();
|
||||
const brain = this.getAttribute("brain");
|
||||
const { spec, ...info } = await fetchSpec(this.endpoint, { dataset: this.dataset, prompt, prior: this._spec, ...(brain ? { brain } : {}) });
|
||||
await this.render(spec, prompt);
|
||||
this.dispatchEvent(new CustomEvent("spec", { detail: { spec, prompt, info } }));
|
||||
} catch (e) {
|
||||
this._msg.textContent = String(e.message || e);
|
||||
}
|
||||
}
|
||||
|
||||
async render(spec, prompt) {
|
||||
await this.load();
|
||||
const v = validateSpec(spec, this._schema);
|
||||
if (!v.ok) throw new Error("That chart didn't pass the checks: " + v.errors.join("; "));
|
||||
const cfg = chartConfig(applySpec(this._data, spec), spec);
|
||||
this._chart?.destroy();
|
||||
this._chart = new globalThis.Chart(this.querySelector("canvas"), cfg);
|
||||
this._spec = spec;
|
||||
this._prompt = prompt;
|
||||
this._msg.textContent = spec.title || "";
|
||||
this.querySelector(".aic-save").hidden = false;
|
||||
}
|
||||
|
||||
async save() {
|
||||
const detail = { spec: this._spec, prompt: this._prompt, dataset: this.dataset };
|
||||
this.dispatchEvent(new CustomEvent("save", { detail }));
|
||||
if (this.onSave) {
|
||||
try { await this.onSave(detail); this._msg.textContent = "Saved."; } catch (e) { this._msg.textContent = String(e.message || e); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (globalThis.customElements && !customElements.get("ai-chart")) customElements.define("ai-chart", AiChart);
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Compare AICharts brains on one fixed question set against the live demo.
|
||||
|
||||
python3 bench.py https://aicharts.yolopreneur.lol [brain ...] (default brains: llm jev clef)
|
||||
python3 bench.py <url> clef@0 Clef with the fallback off (calibration)
|
||||
|
||||
Each question lists what a right chart must have; a spec scores 1 when it has all of it. Reports, per brain: right
|
||||
answers, median and p90 latency (as the browser sees it), and how often a decision brain handed off to the LLM.
|
||||
Uses the public API like a visitor, so it counts against the demo's daily AI limit (12 calls per brain).
|
||||
"""
|
||||
import json
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
Q = [
|
||||
("total revenue by region", {"x": "region", "agg": ("sum", "amount")}),
|
||||
("monthly revenue, web vs store", {"mark": {"line"}, "x": "month", "series": "channel", "agg": ("sum", "amount")}),
|
||||
("share of units by product", {"mark": {"pie", "doughnut"}, "x": "product", "agg": ("sum", "units")}),
|
||||
("top 3 products in the West by revenue", {"x": "product", "only": {"region": "West"}, "agg": ("sum", "amount"), "limit": 3}),
|
||||
("average order amount by channel", {"x": "channel", "agg": ("avg", "amount")}),
|
||||
("how did matcha units trend month by month?", {"mark": {"line"}, "x": "month", "only": {"product": "Matcha"}, "agg": ("sum", "units")}),
|
||||
("how many orders per region", {"x": "region", "agg": ("count", None)}),
|
||||
("biggest single order amount for each product", {"x": "product", "agg": ("max", "amount")}),
|
||||
("store revenue by region", {"x": "region", "only": {"channel": "store"}, "agg": ("sum", "amount")}),
|
||||
("teapot units by month in the North", {"x": "month", "only": {"product": "Teapot", "region": "North"}, "agg": ("sum", "units")}),
|
||||
("which region sells the most hojicha?", {"x": "region", "only": {"product": "Hojicha"}, "sorted_desc": True}),
|
||||
("revenue by product, split by channel", {"x": {"product", "channel"}, "series": {"product", "channel"}, "agg": ("sum", "amount")}),
|
||||
]
|
||||
|
||||
|
||||
def facts(spec):
|
||||
f = {"mark": spec.get("mark"), "x": spec.get("x"), "series": spec.get("series"), "only": {}, "limit": None, "sorted_desc": False}
|
||||
for s in spec.get("transform") or []:
|
||||
if s.get("op") == "filter":
|
||||
if "eq" in s:
|
||||
f["only"][s["field"]] = s["eq"]
|
||||
elif isinstance(s.get("in"), list) and len(s["in"]) == 1:
|
||||
f["only"][s["field"]] = s["in"][0]
|
||||
elif s.get("op") == "group":
|
||||
f["agg"] = (s.get("agg"), None if s.get("agg") == "count" else s.get("field"))
|
||||
elif s.get("op") == "limit":
|
||||
f["limit"] = s.get("n")
|
||||
elif s.get("op") == "sort" and s.get("order") == "desc" and s.get("field") not in ("month", "date", f["x"]):
|
||||
f["sorted_desc"] = True
|
||||
return f
|
||||
|
||||
|
||||
def misses(spec, want):
|
||||
f, out = facts(spec), []
|
||||
for k, v in want.items():
|
||||
got = f.get(k)
|
||||
if k == "only":
|
||||
if {a: str(b).lower() for a, b in got.items()} != {a: b.lower() for a, b in v.items()}:
|
||||
out.append(f"only {got} != {v}")
|
||||
elif isinstance(v, set):
|
||||
if got not in v:
|
||||
out.append(f"{k} {got} not in {sorted(v)}")
|
||||
elif k == "agg":
|
||||
if got is None or got[0] != v[0] or (v[1] and got[1] != v[1]):
|
||||
out.append(f"agg {got} != {v}")
|
||||
elif got != v:
|
||||
out.append(f"{k} {got} != {v}")
|
||||
if "x" in want and "series" in want and f["x"] == f["series"]:
|
||||
out.append("x == series")
|
||||
return out
|
||||
|
||||
|
||||
def ask(url, prompt, brain):
|
||||
name, _, floor = brain.partition("@") # "clef@0" = Clef with no fallback: its own answers, for calibration
|
||||
body = json.dumps({"dataset": "orders", "prompt": prompt, "brain": name, **({"floor": float(floor)} if floor else {})}).encode()
|
||||
req = urllib.request.Request(url.rstrip("/") + "/api/spec", data=body, # Cloudflare turns away Python's default agent
|
||||
headers={"content-type": "application/json", "user-agent": "aicharts-bench/1"})
|
||||
t0 = time.time()
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=90) as r:
|
||||
out = json.loads(r.read())
|
||||
except urllib.error.HTTPError as e:
|
||||
out = json.loads(e.read() or b"{}")
|
||||
return out, int((time.time() - t0) * 1000)
|
||||
|
||||
|
||||
def main():
|
||||
url, brains = sys.argv[1], sys.argv[2:] or ["llm", "jev", "clef"]
|
||||
report = {}
|
||||
for brain in brains:
|
||||
rows = []
|
||||
for prompt, want in Q:
|
||||
out, ms = ask(url, prompt, brain)
|
||||
m = misses(out["spec"], want) if out.get("spec") else [out.get("error", "no spec")]
|
||||
rows.append({"q": prompt, "ok": not m, "ms": ms, "via": out.get("brain"), "handoff": (out.get("handoff") or {}).get("reason"),
|
||||
"confidence": out.get("confidence"), "confidences": out.get("confidences"), "misses": m})
|
||||
print(f"[{brain}] {'ok ' if not m else 'MISS'} {ms:>6} ms via {out.get('brain')} conf {out.get('confidence')}: {prompt}"
|
||||
+ (f" -- {'; '.join(m)}" if m else "")
|
||||
+ (f" (handoff: {rows[-1]['handoff']})" if rows[-1]["handoff"] else ""), flush=True)
|
||||
lat = sorted(r["ms"] for r in rows)
|
||||
report[brain] = {"right": sum(r["ok"] for r in rows), "of": len(rows), "median_ms": int(statistics.median(lat)),
|
||||
"p90_ms": lat[int(0.9 * (len(lat) - 1))], "handoffs": sum(1 for r in rows if r["handoff"]), "rows": rows}
|
||||
print(json.dumps({b: {k: v for k, v in r.items() if k != "rows"} for b, r in report.items()}, indent=1))
|
||||
json.dump(report, open("bench.json", "w"), indent=1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,121 @@
|
||||
// A decision-model brain for AICharts: Jev (TypeSafe) or Clef (Cloudflare) answers a handful of choice questions in
|
||||
// one pass, and this module assembles the chart spec from the answers. The model never writes JSON, so a spec can't
|
||||
// be malformed; the validator still checks it, and weak confidence hands the question to the LLM path.
|
||||
//
|
||||
// questions(info) -> System One questions for one dataset (info = describe(rows, columns))
|
||||
// assemble(answers, info) -> { spec, confidence, weakest, alternatives }
|
||||
// decide(env, model, info, prompt, prior) -> the above plus { model, ms } (throws if the model call fails)
|
||||
|
||||
const MARK_MENU = {
|
||||
bar: "Comparing amounts across categories or a handful of periods.",
|
||||
line: "A value over time (months, dates) or another ordered sequence; the trend matters.",
|
||||
pie: "Shares of one whole across a few categories (6 or fewer).",
|
||||
doughnut: "Shares of one whole, shown as a ring; same use as a pie.",
|
||||
scatter: "Two numeric measures plotted against each other across many points.",
|
||||
};
|
||||
const AGG_MENU = {
|
||||
sum: "The total (revenue, units sold): add the values up.",
|
||||
avg: "The typical or average value per order.",
|
||||
count: "How many orders or rows, regardless of their values.",
|
||||
min: "The smallest value.",
|
||||
max: "The largest value.",
|
||||
};
|
||||
const ORDER_MENU = {
|
||||
natural: "No particular order asked for (time stays in time order).",
|
||||
value_desc: "Biggest first: top, best, most, highest, ranking.",
|
||||
value_asc: "Smallest first: bottom, worst, least, lowest.",
|
||||
};
|
||||
const TOP_MENU = { all: "Show every group.", 3: "Only the top or bottom 3.", 5: "Only the top or bottom 5.", 10: "Only the top or bottom 10." };
|
||||
|
||||
function shuffled(menu) {
|
||||
const e = Object.entries(menu);
|
||||
for (let i = e.length - 1; i > 0; i--) { const j = Math.floor(Math.random() * (i + 1)); [e[i], e[j]] = [e[j], e[i]]; }
|
||||
return Object.fromEntries(e);
|
||||
}
|
||||
const choice = (instructions, menu) => ({ type: "choice", instructions, criteria: shuffled(menu) });
|
||||
|
||||
const describeCol = (c, agg) => {
|
||||
const a = agg[c.name] || {};
|
||||
if (a.values) return `${c.type} column; values: ${a.values.join(", ")}`;
|
||||
if ("min" in a) return `${c.type} column from ${a.min} to ${a.max}`;
|
||||
return `${c.type} column`;
|
||||
};
|
||||
|
||||
export function questions(info) {
|
||||
const cols = info.columns.filter((c) => c.name !== "id");
|
||||
const text = cols.filter((c) => c.type === "string");
|
||||
const nums = cols.filter((c) => c.type === "number");
|
||||
const axis = Object.fromEntries(cols.map((c) => [c.name, describeCol(c, info.aggregates)]));
|
||||
const measure = { ...Object.fromEntries(nums.map((c) => [c.name, describeCol(c, info.aggregates)])), rows: "The number of orders (no column)." };
|
||||
const q = {
|
||||
mark: choice("Which chart type answers the request best?", MARK_MENU),
|
||||
x: choice("Which column goes on the x axis (the categories or time the chart is broken down by)?", axis),
|
||||
measure: choice("Which quantity does the chart measure?", measure),
|
||||
agg: choice("How should that quantity be combined within each group?", AGG_MENU),
|
||||
series: choice("Should the chart split into one coloured series per value of another column? Pick none unless the request compares groups against each other (\"web vs store\", \"by region over time\").",
|
||||
{ none: "One series only.", ...Object.fromEntries(text.map((c) => [c.name, `one series per ${c.name}`])) }),
|
||||
order: choice("How should the groups be ordered?", ORDER_MENU),
|
||||
top: choice("Does the request limit how many groups to show?", TOP_MENU),
|
||||
};
|
||||
for (const c of text) {
|
||||
const vals = info.aggregates[c.name]?.values;
|
||||
if (vals?.length && vals.length <= 30)
|
||||
q[`only.${c.name}`] = choice(`Does the request restrict the data to one ${c.name}? Pick any unless it names one.`,
|
||||
{ any: `No restriction on ${c.name}.`, ...Object.fromEntries(vals.map((v) => [v, `Only ${c.name} = ${v}.`])) });
|
||||
}
|
||||
return q;
|
||||
}
|
||||
|
||||
export function assemble(answers, info) {
|
||||
const pick = (k) => answers[k]?.choice;
|
||||
const conf = Object.fromEntries(Object.entries(answers).map(([k, v]) => [k, Number(v?.confidence ?? 0)]));
|
||||
const types = Object.fromEntries(info.columns.map((c) => [c.name, c.type]));
|
||||
const mark = pick("mark"), x = pick("x"), measure = pick("measure");
|
||||
let agg = pick("agg");
|
||||
let series = pick("series");
|
||||
if (series === "none" || series === x || ["pie", "doughnut", "scatter"].includes(mark)) series = undefined;
|
||||
const gating = ["mark", "x", "measure", "order", "top"];
|
||||
if (measure === "rows") agg = "count"; else gating.push("agg");
|
||||
if (series) gating.push("series");
|
||||
|
||||
const transform = [];
|
||||
for (const k of Object.keys(answers).filter((k) => k.startsWith("only."))) {
|
||||
const v = pick(k);
|
||||
if (v && v !== "any") { transform.push({ op: "filter", field: k.slice(5), eq: v }); gating.push(k); }
|
||||
}
|
||||
let spec;
|
||||
if (mark === "scatter") {
|
||||
// a scatter plots rows, not groups: x and the measure are both raw number columns
|
||||
spec = { mark, x, y: measure === "rows" ? "amount" : measure, transform };
|
||||
} else {
|
||||
transform.push({ op: "group", by: series ? [x, series] : [x], agg, ...(agg === "count" ? {} : { field: measure }), as: "value" });
|
||||
const order = pick("order");
|
||||
if (order === "value_desc" || order === "value_asc") transform.push({ op: "sort", field: "value", order: order === "value_desc" ? "desc" : "asc" });
|
||||
else if (["month", "date", "number"].includes(types[x])) transform.push({ op: "sort", field: x, order: "asc" });
|
||||
const top = pick("top");
|
||||
if (top && top !== "all") transform.push({ op: "limit", n: Number(top) });
|
||||
spec = { mark, x, y: "value", ...(series ? { series } : {}), transform };
|
||||
}
|
||||
const what = measure === "rows" ? "Orders" : `${{ sum: "Total", avg: "Average", count: "Count of", min: "Lowest", max: "Highest" }[agg]} ${measure}`;
|
||||
const only = transform.filter((s) => s.op === "filter").map((s) => s.eq);
|
||||
spec.title = mark === "scatter" ? `${spec.y} vs ${x}` : `${what} by ${x}${series ? ` and ${series}` : ""}${only.length ? ` (${only.join(", ")})` : ""}`;
|
||||
|
||||
const weakestOn = gating.reduce((lo, k) => ((conf[k] ?? 0) < (conf[lo] ?? 0) ? k : lo), gating[0]);
|
||||
const weakest = conf[weakestOn] ?? 0;
|
||||
// other chart types the model rated close behind: cheap to offer, already the same data
|
||||
const probs = answers.mark?.probabilities || {};
|
||||
const alternatives = Object.entries(probs).filter(([m, p]) => m !== mark && p >= 0.15).sort((a, b) => b[1] - a[1]).slice(0, 2)
|
||||
.map(([m, p]) => ({ mark: m, p: Math.round(p * 100) / 100 }));
|
||||
return { spec, confidence: conf, weakest: Math.round(weakest * 100) / 100, weakestOn, alternatives };
|
||||
}
|
||||
|
||||
export async function decide(env, model, info, prompt, prior) {
|
||||
const state = { user_request: prompt, table: { columns: info.columns.map((c) => c.name), rows: info.rows },
|
||||
...(prior ? { current_chart: prior } : {}) };
|
||||
const input = { state, questions: questions(info) };
|
||||
if (model.startsWith("@cf/cloudflare/clef")) input.model = model.endsWith("flash") ? "clef-flash" : "clef";
|
||||
const t0 = Date.now();
|
||||
const res = await env.AI.run(model, input);
|
||||
if (!res?.answers) throw new Error("the decision model returned no answers");
|
||||
return { ...assemble(res.answers, info), model: res.model || model, ms: Date.now() - t0 };
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Deploy the AICharts demo: one Worker (`time-aicharts`), one D1 database, Workers AI.
|
||||
|
||||
python3 .github/time/labs/aicharts/deploy.py # prints {url, worker, db, rev}
|
||||
|
||||
Reuses cloudflare.py. Lives where ventures live: aicharts.<TIME_VENTURE_DOMAIN> when that is set (a Workers custom
|
||||
domain; Cloudflare adds the record and the cert), the account's workers.dev otherwise. Idempotent: the same database is
|
||||
reused and the script is replaced in place.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE.parents[1]))
|
||||
import cloudflare as cf # noqa: E402
|
||||
|
||||
NAME = "time-aicharts"
|
||||
MAIN = b'import src from "./aicharts.txt";\nimport { make } from "./worker.js";\nexport default make(src);\n'
|
||||
|
||||
|
||||
def parts() -> list[tuple[str, str, bytes]]:
|
||||
lib = (HERE / "aicharts.js").read_bytes()
|
||||
return [("main.js", "application/javascript+module", MAIN),
|
||||
("worker.js", "application/javascript+module", (HERE / "worker.js").read_bytes()),
|
||||
("aicharts.js", "application/javascript+module", lib),
|
||||
("decide.js", "application/javascript+module", (HERE / "decide.js").read_bytes()),
|
||||
("aicharts.txt", "text/plain", lib)]
|
||||
|
||||
|
||||
def deploy() -> dict:
|
||||
if not cf.enabled():
|
||||
sys.exit("CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID are needed")
|
||||
code = parts()
|
||||
rev = hashlib.sha256(b"".join(d for _, _, d in code)).hexdigest()[:12]
|
||||
vars_ = {"REV": rev, "AI_MODEL": cf.env("AICHARTS_AI_MODEL", "@cf/meta/llama-3.3-70b-instruct-fp8-fast"), "AI_DAILY": cf.env("AICHARTS_AI_DAILY", "300"),
|
||||
"SAVE_DAILY": cf.env("AICHARTS_SAVE_DAILY", "1000"),
|
||||
"BRAIN": cf.env("AICHARTS_BRAIN", "llm"), "FLOOR": cf.env("AICHARTS_FLOOR", "0.5")}
|
||||
db = cf.database("aicharts") # named time-aicharts, like the script
|
||||
meta = {"main_module": "main.js", "compatibility_date": cf.COMPAT,
|
||||
"bindings": [{"type": "d1", "name": "DB", "id": db}, {"type": "ai", "name": "AI"}]
|
||||
+ [{"type": "plain_text", "name": k, "text": str(v)} for k, v in vars_.items()]}
|
||||
data, ctype = cf.multipart([("metadata", "application/json", json.dumps(meta).encode()), *code])
|
||||
cf.cf("PUT", f"{cf.account()}/workers/scripts/{NAME}", data=data, ctype=ctype)
|
||||
hostname = cf.host("aicharts")
|
||||
return {"url": cf.attach(NAME, hostname) if hostname else cf.workers_dev(NAME), "worker": NAME, "db": db, "rev": rev}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(json.dumps(deploy(), indent=1))
|
||||
@@ -0,0 +1,250 @@
|
||||
// Offline test for the AICharts library and demo Worker: a real SQLite behind a D1-shaped shim, and a fake Workers AI.
|
||||
// Run: node .github/time/labs/aicharts/test_aicharts.mjs
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { DatabaseSync } from "node:sqlite";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const lib = await import(join(here, "aicharts.js"));
|
||||
const W = await import(join(here, "worker.js"));
|
||||
const src = readFileSync(join(here, "aicharts.js"), "utf8");
|
||||
|
||||
// ---------- library ----------
|
||||
const schema = { columns: [{ name: "month", type: "month" }, { name: "region", type: "string" }, { name: "amount", type: "number" }] };
|
||||
const rows = [
|
||||
{ month: "2026-01", region: "West", amount: 10 }, { month: "2026-01", region: "East", amount: 5 },
|
||||
{ month: "2026-02", region: "West", amount: 7 }, { month: "2026-02", region: "East", amount: 9 },
|
||||
];
|
||||
const bySum = { mark: "bar", x: "region", y: "value",
|
||||
transform: [{ op: "group", by: ["region"], agg: "sum", field: "amount", as: "value" }, { op: "sort", field: "value", order: "desc" }] };
|
||||
assert.ok(lib.validateSpec(bySum, schema).ok);
|
||||
assert.deepEqual(lib.applySpec(rows, bySum), [{ region: "West", value: 17 }, { region: "East", value: 14 }]);
|
||||
|
||||
const bad = lib.validateSpec({ mark: "radar", x: "month", y: "amount",
|
||||
transform: [{ op: "group", by: ["region"], agg: "sum", field: "amount" }, { op: "eval", code: "alert(1)" }] }, schema);
|
||||
assert.equal(bad.ok, false);
|
||||
assert.ok(bad.errors.some((e) => e.includes("mark")));
|
||||
assert.ok(bad.errors.some((e) => e.includes("step 2")));
|
||||
assert.ok(bad.errors.some((e) => e.startsWith("x must be a field the transform leaves")), "month is gone after the group");
|
||||
assert.equal(lib.validateSpec({ mark: "bar", x: "region", y: "region" }, schema).ok, false, "y must be a number");
|
||||
assert.equal(lib.validateSpec({ mark: "bar", x: "region", y: "amount", transform: Array(9).fill({ op: "limit", n: 1 }) }, schema).ok, false);
|
||||
assert.equal(lib.validateSpec("[1]", schema).ok, false);
|
||||
|
||||
const lines = { mark: "line", x: "month", y: "value", series: "region",
|
||||
transform: [{ op: "group", by: ["month", "region"], agg: "sum", field: "amount", as: "value" }, { op: "sort", field: "month" }] };
|
||||
const cfg = lib.chartConfig(lib.applySpec(rows, lines), lines);
|
||||
assert.deepEqual(cfg.data.labels, ["2026-01", "2026-02"]);
|
||||
assert.deepEqual(cfg.data.datasets.map((d) => [d.label, d.data]), [["West", [10, 7]], ["East", [5, 9]]]);
|
||||
const pie = lib.chartConfig(lib.applySpec(rows, { ...bySum, mark: "pie" }), { ...bySum, mark: "pie" });
|
||||
assert.equal(pie.data.datasets[0].backgroundColor.length, 2);
|
||||
const binned = lib.applySpec(rows, { transform: [{ op: "bin", field: "amount", step: 5 }, { op: "filter", field: "amount_bin", gte: 5, lte: 5 }] });
|
||||
assert.deepEqual(binned.map((r) => r.amount), [5, 7, 9]);
|
||||
|
||||
// attach() drives an existing chart and keeps the rows in the browser
|
||||
const sent = [];
|
||||
globalThis.fetch = async (url, init) => { sent.push({ url, body: JSON.parse(init.body) }); return Response.json({ spec: bySum }); };
|
||||
const chart = { config: { type: "line", data: {}, options: { responsive: true } }, updates: 0, update() { this.updates++; } };
|
||||
await lib.attach(chart, { data: rows, schema, endpoint: "https://x.example/", dataset: "orders" }).ask("sales by region");
|
||||
assert.equal(sent[0].url, "https://x.example/api/spec");
|
||||
assert.deepEqual(Object.keys(sent[0].body).sort(), ["dataset", "prompt"], "no rows go to the server");
|
||||
assert.equal(chart.config.type, "bar");
|
||||
assert.equal(chart.config.options.responsive, true);
|
||||
assert.equal(chart.updates, 1);
|
||||
await lib.attach(chart, { data: rows, schema, endpoint: "https://x.example", dataset: "orders" }).ask("now as a pie");
|
||||
const a = lib.attach(chart, { data: rows, schema, endpoint: "https://x.example", dataset: "orders" });
|
||||
await a.ask("one"); await a.ask("two");
|
||||
assert.deepEqual(sent.at(-1).body.prior, bySum, "follow-ups carry the last chart");
|
||||
|
||||
// ---------- worker ----------
|
||||
assert.deepEqual(W.firstObject('Sure!\n```json\n{"a": {"b": "}"}}\n```'), { a: { b: "}" } });
|
||||
assert.deepEqual(W.firstObject('Sure! {"mark": "bar", "x": "a"} trailing {'), { mark: "bar", x: "a" });
|
||||
assert.equal(W.firstObject("no json"), null);
|
||||
assert.deepEqual(W.normalizeSpec({ mark: "Bar", x: "region", y: "value",
|
||||
transform: [{ type: "groupBy", groupBy: "region", aggregate: "SUM", field: "amount", as: "value" }, { op: "sort", field: "value", order: "DESC" }] }),
|
||||
{ mark: "bar", x: "region", y: "value", transform: [{ op: "group", by: ["region"], agg: "sum", field: "amount", as: "value" }, { op: "sort", field: "value", order: "desc" }] });
|
||||
assert.equal(W.normalizeSpec({ mark: "bar", transform: [{ op: "group", agg: "mean" }] }).transform[0].agg, "avg");
|
||||
|
||||
class D1 {
|
||||
constructor() { this.db = new DatabaseSync(":memory:"); this.writes = []; }
|
||||
prepare(sql) {
|
||||
const db = this;
|
||||
const make = (args) => ({
|
||||
bind: (...a) => make(a),
|
||||
async all() { db.log(sql); return { results: db.db.prepare(sql).all(...args) }; },
|
||||
async first() { db.log(sql); return db.db.prepare(sql).get(...args) ?? null; },
|
||||
async run() { db.log(sql); const s = db.db.prepare(sql); return /RETURNING|^\s*SELECT/i.test(sql) ? { results: s.all(...args) } : (s.run(...args), { results: [] }); },
|
||||
});
|
||||
return make([]);
|
||||
}
|
||||
log(sql) { if (/^\s*(INSERT|UPDATE|DELETE|CREATE)/i.test(sql)) this.writes.push(sql.split("(")[0].trim()); }
|
||||
async batch(stmts) { return Promise.all(stmts.map((s) => s.run())); }
|
||||
}
|
||||
|
||||
const answers = [];
|
||||
const calls = [];
|
||||
const env = { DB: new D1(), REV: "r1", AI_MODEL: "@cf/test/m", AI_DAILY: "5", SAVE_DAILY: "100",
|
||||
AI: { async run(model, input) {
|
||||
if (input.questions) { decisions.push({ model, input }); const d = decided.shift(); if (d instanceof Error) throw d; return d; }
|
||||
calls.push(input); const a = answers.shift(); if (a instanceof Error) throw a; return a; } } };
|
||||
const decisions = [], decided = [];
|
||||
const worker = W.make(src);
|
||||
const call = (path, init = {}, ip = "1.1.1.1") =>
|
||||
worker.fetch(new Request("https://aicharts.t.workers.dev" + path, { ...init, headers: { "cf-connecting-ip": ip, ...(init.headers || {}) } }), env);
|
||||
const post = (path, body, ip) => call(path, { method: "POST", body: JSON.stringify(body), headers: { "content-type": "application/json" } }, ip);
|
||||
|
||||
let r = await call("/");
|
||||
let html = await r.text();
|
||||
assert.ok(html.includes("<ai-chart") && html.includes("https://aicharts.t.workers.dev/aicharts.js"));
|
||||
r = await call("/aicharts.js");
|
||||
assert.equal(await r.text(), src);
|
||||
assert.equal(r.headers.get("access-control-allow-origin"), "*");
|
||||
|
||||
// the schema: what the model sees. Seeded once, deterministic.
|
||||
r = await call("/api/schema?dataset=orders");
|
||||
const sch = await r.json();
|
||||
assert.equal(sch.sample.length, 5);
|
||||
assert.deepEqual(sch.aggregates.region.values, ["East", "North", "South", "West"]);
|
||||
assert.equal(sch.schema.columns.find((c) => c.name === "amount").type, "number");
|
||||
const data = (await (await call("/api/data?dataset=orders")).json()).rows;
|
||||
assert.equal(data.length, 24 * 4 * 5);
|
||||
assert.deepEqual(data, W.seedOrders());
|
||||
assert.equal((await call("/api/schema?dataset=users")).status, 404);
|
||||
const seeded = env.DB.writes.filter((w) => w.startsWith("INSERT OR IGNORE INTO orders")).length;
|
||||
await call("/api/data");
|
||||
assert.equal(env.DB.writes.filter((w) => w.startsWith("INSERT OR IGNORE INTO orders")).length, seeded, "seeding happens once");
|
||||
|
||||
// a spec: plain prompt; the system prompt carries schema + sample, never all rows
|
||||
const good = { title: "Revenue by region", mark: "bar", x: "region", y: "value",
|
||||
transform: [{ op: "group", by: ["region"], agg: "sum", field: "amount", as: "value" }] };
|
||||
answers.push({ response: good });
|
||||
r = await post("/api/spec", { dataset: "orders", prompt: "revenue by region" });
|
||||
assert.equal(r.status, 200);
|
||||
assert.deepEqual((await r.json()).spec, good);
|
||||
assert.equal(calls.length, 1);
|
||||
assert.equal(calls[0].response_format, undefined, "plain prompt first");
|
||||
const sys = calls[0].messages[0].content;
|
||||
assert.ok(sys.includes("region (string)") && sys.includes('"North"'));
|
||||
assert.ok(sys.length < 4000 && !sys.includes(`"id":${data.length}`), "the prompt has a sample, not the table");
|
||||
|
||||
// no JSON at all -> JSON mode; prose around JSON is fine; a bad answer gets one repair round with the errors
|
||||
answers.push({ response: "I think a line chart would be nice." },
|
||||
{ response: '{"mark": "bar", "x": "month", "y": "amount", "transform": [{"op": "group", "by": ["region"], "agg": "sum", "field": "amount"}]}' },
|
||||
{ choices: [{ message: { content: "Fixed: " + JSON.stringify({ ...good, mark: "line", x: "region" }) } }] });
|
||||
calls.length = 0;
|
||||
r = await post("/api/spec", { dataset: "orders", prompt: "trend", prior: good });
|
||||
assert.equal(r.status, 200);
|
||||
assert.equal((await r.json()).spec.mark, "line");
|
||||
assert.deepEqual(calls.map((c) => c.response_format?.type ?? null), [null, "json_object", null]);
|
||||
assert.ok(calls[0].messages.some((m) => m.content.includes("Current chart spec")), "the prior chart is context");
|
||||
assert.ok(calls.at(-1).messages.at(-1).content.includes("fails these checks"), "one repair round with the errors");
|
||||
|
||||
// two bad answers: a 422 with the reasons, nothing drawn
|
||||
answers.push({ response: "{}" }, { response: '{"mark": "pie"}' });
|
||||
r = await post("/api/spec", { dataset: "orders", prompt: "nonsense" });
|
||||
assert.equal(r.status, 422);
|
||||
const failed = await r.json();
|
||||
assert.match(failed.error, /valid chart/);
|
||||
assert.equal(failed.model_said, '{"mark": "pie"}', "the model's answer is shown on failure");
|
||||
|
||||
// limits: AI_DAILY=5 overall; we've used 3
|
||||
answers.push({ response: good }, { response: good });
|
||||
assert.equal((await post("/api/spec", { prompt: "a" }, "2.2.2.2")).status, 200);
|
||||
assert.equal((await post("/api/spec", { prompt: "b" }, "2.2.2.2")).status, 200);
|
||||
assert.equal((await post("/api/spec", { prompt: "c" }, "3.3.3.3")).status, 429);
|
||||
assert.equal((await post("/api/spec", { prompt: "" })).status, 400);
|
||||
|
||||
// ---------- decision brains (Jev / Clef): choice answers -> spec, LLM only as the door ----------
|
||||
const D = await import(join(here, "decide.js"));
|
||||
const ans = (choice, confidence = 0.9, probabilities) => ({ type: "choice", choice, confidence, probabilities: probabilities || { [choice]: confidence } });
|
||||
const westTop3 = { mark: ans("bar", 0.8, { bar: 0.8, pie: 0.17, line: 0.03 }), x: ans("product"), measure: ans("amount"), agg: ans("sum"), series: ans("none"),
|
||||
order: ans("value_desc"), top: ans("3"), "only.region": ans("West", 0.95), "only.product": ans("any"), "only.channel": ans("any") };
|
||||
const infoNow = W.describe(data, sch.schema.columns);
|
||||
const qs = D.questions(infoNow);
|
||||
assert.deepEqual(Object.keys(qs).sort(), ["agg", "mark", "measure", "only.channel", "only.product", "only.region", "order", "series", "top", "x"]);
|
||||
assert.ok(!("id" in qs.x.criteria), "id is never an axis");
|
||||
assert.ok("rows" in qs.measure.criteria && "amount" in qs.measure.criteria && !("region" in qs.measure.criteria));
|
||||
assert.deepEqual(Object.keys(qs["only.region"].criteria).sort(), ["East", "North", "South", "West", "any"]);
|
||||
const asm = D.assemble(westTop3, infoNow);
|
||||
assert.deepEqual(asm.spec.transform, [{ op: "filter", field: "region", eq: "West" },
|
||||
{ op: "group", by: ["product"], agg: "sum", field: "amount", as: "value" }, { op: "sort", field: "value", order: "desc" }, { op: "limit", n: 3 }]);
|
||||
assert.equal(asm.spec.title, "Total amount by product (West)");
|
||||
assert.ok(lib.validateSpec(asm.spec, sch.schema).ok);
|
||||
assert.equal(lib.applySpec(data, asm.spec).length, 3);
|
||||
assert.deepEqual(asm.alternatives, [{ mark: "pie", p: 0.17 }]);
|
||||
assert.equal(asm.weakest, 0.8);
|
||||
// time on x sorts by time; a series only when asked; count needs no field
|
||||
const monthly = D.assemble({ ...westTop3, mark: ans("line"), x: ans("month"), measure: ans("rows"), agg: ans("avg", 0.1), series: ans("channel"),
|
||||
order: ans("natural"), top: ans("all"), "only.region": ans("any") }, infoNow);
|
||||
assert.deepEqual(monthly.spec.transform, [{ op: "group", by: ["month", "channel"], agg: "count", as: "value" }, { op: "sort", field: "month", order: "asc" }]);
|
||||
assert.equal(monthly.spec.series, "channel");
|
||||
assert.equal(monthly.weakest, 0.9, "agg doesn't gate when counting rows");
|
||||
assert.ok(lib.validateSpec(monthly.spec, sch.schema).ok);
|
||||
assert.equal(D.assemble({ ...westTop3, mark: ans("pie"), series: ans("channel") }, infoNow).spec.series, undefined, "no series on a pie");
|
||||
|
||||
// through the worker: a confident Jev answer is the spec, no LLM call
|
||||
const resetQuota = () => env.DB.db.exec("DELETE FROM quota");
|
||||
resetQuota(); calls.length = 0;
|
||||
decided.push({ model: "jev-1", answers: westTop3, usage: {} });
|
||||
r = await post("/api/spec", { dataset: "orders", prompt: "top 3 products in the West by revenue", brain: "jev" });
|
||||
let body = await r.json();
|
||||
assert.equal(r.status, 200, JSON.stringify(body));
|
||||
assert.equal(body.brain, "jev");
|
||||
assert.equal(decisions.at(-1).model, "typesafe/jev");
|
||||
assert.equal(decisions.at(-1).input.state.user_request, "top 3 products in the West by revenue");
|
||||
const toJev = JSON.stringify(decisions.at(-1).input), mid = data[200];
|
||||
assert.ok(!toJev.includes(mid.date) && !toJev.includes(String(mid.amount)) && !("sample" in decisions.at(-1).input.state),
|
||||
"rows never reach the decision model");
|
||||
assert.equal(calls.length, 0);
|
||||
assert.deepEqual(body.spec, asm.spec);
|
||||
// Clef names its variant in the input
|
||||
decided.push({ answers: westTop3 });
|
||||
await post("/api/spec", { dataset: "orders", prompt: "x", brain: "clef" });
|
||||
assert.equal(decisions.at(-1).model, "@cf/cloudflare/clef-flash");
|
||||
assert.equal(decisions.at(-1).input.model, "clef-flash");
|
||||
// unsure -> the door: the LLM answers, and the response says why
|
||||
decided.push({ answers: { ...westTop3, x: ans("product", 0.31) } });
|
||||
answers.push({ response: good });
|
||||
body = await (await post("/api/spec", { dataset: "orders", prompt: "hmm", brain: "jev" })).json();
|
||||
assert.equal(body.brain, "llm");
|
||||
assert.match(body.handoff.reason, /low confidence \(0.31 on x\)/);
|
||||
assert.equal(body.handoff.confidences.x, 0.31);
|
||||
assert.deepEqual(body.spec, good);
|
||||
// the same answer with the caller's own lower bar is accepted, with every confidence shown
|
||||
decided.push({ answers: { ...westTop3, x: ans("product", 0.31) } });
|
||||
body = await (await post("/api/spec", { dataset: "orders", prompt: "hmm", brain: "jev", floor: 0.2 })).json();
|
||||
assert.equal(body.brain, "jev");
|
||||
assert.equal(body.confidences.x, 0.31);
|
||||
// decision model down -> the door too
|
||||
decided.push(new Error("no such model"));
|
||||
answers.push({ response: good });
|
||||
body = await (await post("/api/spec", { dataset: "orders", prompt: "hmm", brain: "jev" })).json();
|
||||
assert.match(body.handoff.reason, /decision model error: no such model/);
|
||||
// the server default brain, and an unknown brain falls back to it
|
||||
resetQuota(); env.BRAIN = "jev"; decided.push({ answers: westTop3 });
|
||||
assert.equal((await (await post("/api/spec", { prompt: "q", brain: "gpt" })).json()).brain, "jev");
|
||||
delete env.BRAIN; resetQuota();
|
||||
|
||||
// save, list, load; only valid specs are stored
|
||||
r = await post("/api/charts", { dataset: "orders", prompt: "revenue by region", spec: good });
|
||||
assert.equal(r.status, 201);
|
||||
const { id, link } = await r.json();
|
||||
assert.equal(link, `https://aicharts.t.workers.dev/?c=${id}`);
|
||||
assert.equal((await post("/api/charts", { dataset: "orders", spec: { mark: "bar", x: "nope", y: "amount" } })).status, 400);
|
||||
const list = (await (await call("/api/charts")).json()).charts;
|
||||
assert.deepEqual(list.map((c) => [c.id, c.title]), [[id, "Revenue by region"]]);
|
||||
const back = await (await call("/api/charts/" + id)).json();
|
||||
assert.deepEqual(back.spec, good);
|
||||
assert.equal((await call("/api/charts/zzzzzz")).status, 404);
|
||||
|
||||
// read-only: the only tables written after seeding are quota and saved_charts
|
||||
const after = env.DB.writes.slice(env.DB.writes.lastIndexOf("INSERT OR IGNORE INTO orders") + 1);
|
||||
assert.ok(after.every((w) => /^(CREATE TABLE IF NOT EXISTS|INSERT INTO quota|INSERT INTO saved_charts)/.test(w)), after.join(", "));
|
||||
|
||||
// the saved spec renders with the library against the real rows
|
||||
const out = lib.applySpec(data, back.spec);
|
||||
assert.deepEqual(out.map((x) => x.region).sort(), ["East", "North", "South", "West"]);
|
||||
assert.equal(Math.round(out.reduce((a, x) => a + x.value, 0)), Math.round(data.reduce((a, x) => a + x.amount, 0)));
|
||||
|
||||
console.log("aicharts: all good");
|
||||
@@ -0,0 +1,358 @@
|
||||
// AICharts demo Worker: the page, the library, a sample dataset in D1, the spec endpoint (Workers AI) and saved charts.
|
||||
//
|
||||
// GET / the demo page (Chart.js + <ai-chart>)
|
||||
// GET /aicharts.js the library, as served to any page that embeds it
|
||||
// GET /api/schema?dataset=orders columns, a few sample rows and aggregates: all the model ever sees
|
||||
// GET /api/data?dataset=orders the rows, for the browser to chart
|
||||
// POST /api/spec { dataset, prompt, prior? } -> { spec } a validated chart spec, never code
|
||||
// POST /api/charts { dataset, prompt, spec } -> { id, link } save a chart
|
||||
// GET /api/charts, /api/charts/:id the saved charts
|
||||
//
|
||||
// Read-only except saved_charts and quota. Bindings: DB (D1), AI (Workers AI). Vars: AI_MODEL, AI_DAILY, SAVE_DAILY, REV.
|
||||
// deploy.py uploads main.js, which hands this module the library source: export default make(src).
|
||||
import { validateSpec, MARKS, AGGS } from "./aicharts.js";
|
||||
import { decide } from "./decide.js";
|
||||
|
||||
const SCHEMA = [
|
||||
"CREATE TABLE IF NOT EXISTS orders (id INTEGER PRIMARY KEY, date TEXT, month TEXT, region TEXT, product TEXT, channel TEXT, units INTEGER, amount REAL)",
|
||||
"CREATE TABLE IF NOT EXISTS quota (k TEXT PRIMARY KEY, n INTEGER NOT NULL)",
|
||||
"CREATE TABLE IF NOT EXISTS saved_charts (id TEXT PRIMARY KEY, created TEXT NOT NULL, dataset TEXT NOT NULL, prompt TEXT, title TEXT, spec TEXT NOT NULL)",
|
||||
];
|
||||
const COLUMNS = [
|
||||
{ name: "id", type: "number" }, { name: "date", type: "date" }, { name: "month", type: "month" },
|
||||
{ name: "region", type: "string" }, { name: "product", type: "string" }, { name: "channel", type: "string" },
|
||||
{ name: "units", type: "number" }, { name: "amount", type: "number" },
|
||||
];
|
||||
const DATASETS = { orders: { title: "Orders (sample shop, 24 months)", columns: COLUMNS } };
|
||||
const MAX_PROMPT = 400;
|
||||
|
||||
// The sample shop: deterministic, so every deploy seeds the same rows and tests can count on them.
|
||||
export function seedOrders() {
|
||||
let s = 42;
|
||||
const rnd = () => { s = (s + 0x6d2b79f5) | 0; let t = Math.imul(s ^ (s >>> 15), 1 | s); t ^= t + Math.imul(t ^ (t >>> 7), 61 | t); return ((t ^ (t >>> 14)) >>> 0) / 4294967296; };
|
||||
const regions = ["North", "South", "East", "West"], channels = ["web", "store"];
|
||||
const products = [["Matcha", 24], ["Sencha", 14], ["Hojicha", 12], ["Teapot", 48], ["Whisk", 18]];
|
||||
const rows = [];
|
||||
for (let m = 0; m < 24; m++) {
|
||||
const y = 2024 + Math.floor((m + 9) / 12), mo = ((m + 9) % 12) + 1, month = `${y}-${String(mo).padStart(2, "0")}`;
|
||||
const season = 1 + 0.25 * Math.sin((mo / 12) * 2 * Math.PI) + m * 0.015;
|
||||
for (const region of regions) for (const [product, price] of products) {
|
||||
const units = Math.max(1, Math.round((8 + rnd() * 30) * season * (region === "West" ? 1.3 : 1)));
|
||||
const channel = rnd() < (product === "Teapot" ? 0.4 : 0.65) ? "web" : "store";
|
||||
const day = String(1 + Math.floor(rnd() * 28)).padStart(2, "0");
|
||||
rows.push({ id: rows.length + 1, date: `${month}-${day}`, month, region, product, channel, units,
|
||||
amount: Math.round(units * price * (0.9 + rnd() * 0.2) * 100) / 100 });
|
||||
}
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
let ready = null;
|
||||
async function setup(env) {
|
||||
ready ??= (async () => {
|
||||
await env.DB.batch(SCHEMA.map((q) => env.DB.prepare(q)));
|
||||
const n = await env.DB.prepare("SELECT COUNT(*) AS n FROM orders").first();
|
||||
if (!n?.n) {
|
||||
const ins = env.DB.prepare("INSERT OR IGNORE INTO orders (id, date, month, region, product, channel, units, amount) VALUES (?, ?, ?, ?, ?, ?, ?, ?)");
|
||||
const rows = seedOrders();
|
||||
for (let i = 0; i < rows.length; i += 100)
|
||||
await env.DB.batch(rows.slice(i, i + 100).map((r) => ins.bind(r.id, r.date, r.month, r.region, r.product, r.channel, r.units, r.amount)));
|
||||
}
|
||||
})().catch((e) => { ready = null; throw e; });
|
||||
return ready;
|
||||
}
|
||||
|
||||
const CORS = { "access-control-allow-origin": "*", "access-control-allow-methods": "GET, POST, OPTIONS",
|
||||
"access-control-allow-headers": "content-type" };
|
||||
const json = (body, status = 200) => new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json; charset=utf-8", ...CORS } });
|
||||
|
||||
async function who(req) {
|
||||
const ip = req.headers.get("cf-connecting-ip") || "?";
|
||||
const raw = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(`aicharts:${ip}`));
|
||||
return [...new Uint8Array(raw).slice(0, 8)].map((b) => b.toString(16).padStart(2, "0")).join("");
|
||||
}
|
||||
|
||||
// One counter per (day, kind, person) and one per (day, kind): over either limit means no.
|
||||
async function spend(env, kind, person, perPerson, total) {
|
||||
const day = new Date().toISOString().slice(0, 10);
|
||||
const bump = (k) => env.DB.prepare("INSERT INTO quota (k, n) VALUES (?, 1) ON CONFLICT(k) DO UPDATE SET n = n + 1 RETURNING n").bind(k);
|
||||
const [mine, all] = await env.DB.batch([bump(`${day}:${kind}:${person}`), bump(`${day}:${kind}:*`)]);
|
||||
return mine.results[0].n <= perPerson && all.results[0].n <= total;
|
||||
}
|
||||
|
||||
async function rows(env) {
|
||||
return (await env.DB.prepare("SELECT id, date, month, region, product, channel, units, amount FROM orders ORDER BY id").all()).results;
|
||||
}
|
||||
|
||||
// What the model sees: columns, five rows, distinct values of text columns, ranges of number columns.
|
||||
export function describe(data, columns) {
|
||||
const aggregates = {};
|
||||
for (const c of columns) {
|
||||
const vals = data.map((r) => r[c.name]);
|
||||
if (c.type === "number") aggregates[c.name] = { min: Math.min(...vals), max: Math.max(...vals) };
|
||||
else {
|
||||
const d = [...new Set(vals)].sort();
|
||||
aggregates[c.name] = d.length <= 20 ? { values: d } : { first: d[0], last: d.at(-1), distinct: d.length };
|
||||
}
|
||||
}
|
||||
return { columns, sample: data.slice(0, 5), aggregates, rows: data.length };
|
||||
}
|
||||
|
||||
export function systemPrompt(dataset, info) {
|
||||
return `You turn a question about a table into a chart spec. Answer with ONE JSON object and nothing else.
|
||||
Table "${dataset}" (${info.rows} rows). Columns: ${info.columns.map((c) => `${c.name} (${c.type})`).join(", ")}.
|
||||
Values: ${JSON.stringify(info.aggregates)}
|
||||
Sample rows: ${JSON.stringify(info.sample)}
|
||||
|
||||
Spec: {"title": text, "mark": ${MARKS.map((m) => `"${m}"`).join(" | ")}, "transform": [steps, applied in order, at most 8],
|
||||
"x": field, "y": number field, "series": optional field (one line/bar colour per value)}
|
||||
Steps:
|
||||
{"op": "filter", "field": f, "eq": v} or "in": [v, ...] or "gte": v and/or "lte": v
|
||||
{"op": "group", "by": [up to 2 fields], "agg": ${AGGS.map((a) => `"${a}"`).join(" | ")}, "field": number field (not for count), "as": "value"}
|
||||
after a group, only the "by" fields and "as" remain
|
||||
{"op": "sort", "field": f, "order": "asc" | "desc"}
|
||||
{"op": "bin", "field": number field, "step": n, "as": name}
|
||||
{"op": "limit", "n": 1..500}
|
||||
Use "month" for time on x and sort it asc. Use "series" with group by [x, series] to compare. Pie only for shares of a whole.
|
||||
Example: {"title": "Revenue by region", "mark": "bar", "transform": [{"op": "group", "by": ["region"], "agg": "sum", "field": "amount", "as": "value"}, {"op": "sort", "field": "value", "order": "desc"}], "x": "region", "y": "value"}`;
|
||||
}
|
||||
|
||||
// The first balanced {...} in a model's answer (models wrap JSON in prose or fences when they feel like it).
|
||||
export function firstObject(text) {
|
||||
if (text && typeof text === "object") return text;
|
||||
const s = String(text ?? "");
|
||||
for (let i = s.indexOf("{"); i >= 0; i = s.indexOf("{", i + 1)) {
|
||||
let depth = 0, str = false, esc = false;
|
||||
for (let j = i; j < s.length; j++) {
|
||||
const ch = s[j];
|
||||
if (str) { if (esc) esc = false; else if (ch === "\\") esc = true; else if (ch === '"') str = false; continue; }
|
||||
if (ch === '"') str = true;
|
||||
else if (ch === "{") depth++;
|
||||
else if (ch === "}" && --depth === 0) {
|
||||
try { return JSON.parse(s.slice(i, j + 1)); } catch { break; }
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Small models drift on spelling, not meaning: "SUM", "mean", "aggregate", "groupBy": "region". Fix the spelling, then
|
||||
// the validator decides. Nothing here invents a field or an operation.
|
||||
const AGG_ALIAS = { average: "avg", mean: "avg", total: "sum", minimum: "min", maximum: "max", cnt: "count" };
|
||||
export function normalizeSpec(spec) {
|
||||
if (!spec || typeof spec !== "object" || Array.isArray(spec)) return spec;
|
||||
const low = (v) => (typeof v === "string" ? v.trim().toLowerCase() : v);
|
||||
const out = { ...spec, mark: low(spec.mark) };
|
||||
if (spec.chart && !spec.mark) out.mark = low(spec.chart.type ?? spec.chart);
|
||||
if (Array.isArray(spec.transform)) out.transform = spec.transform.map((s0) => {
|
||||
if (!s0 || typeof s0 !== "object") return s0;
|
||||
const s = { ...s0, op: low(s0.op ?? s0.type) };
|
||||
if (s.op === "groupby" || s.op === "aggregate") s.op = "group";
|
||||
if (s.aggregate != null && s.agg == null) s.agg = s.aggregate;
|
||||
if (s.groupby != null && s.by == null) s.by = s.groupby;
|
||||
if (s.groupBy != null && s.by == null) s.by = s.groupBy;
|
||||
if (typeof s.by === "string") s.by = [s.by];
|
||||
if (s.agg != null) { s.agg = low(s.agg); s.agg = AGG_ALIAS[s.agg] ?? s.agg; }
|
||||
if (s.order != null) s.order = low(s.order);
|
||||
delete s.type; delete s.aggregate; delete s.groupby; delete s.groupBy;
|
||||
return s;
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
// Plain prompt first: the prompt's grammar and example steer small models better than a constrained decoder does (a
|
||||
// one-size schema for every step made them drop agg and by). JSON mode only when no object came back at all.
|
||||
async function model(env, messages) {
|
||||
const input = { messages, max_tokens: 900, temperature: 0.1 };
|
||||
const text = (out) => out?.response ?? out?.choices?.[0]?.message?.content ?? "";
|
||||
let raw = text(await env.AI.run(env.AI_MODEL, input));
|
||||
if (!firstObject(raw)) {
|
||||
try { raw = text(await env.AI.run(env.AI_MODEL, { ...input, response_format: { type: "json_object" } })); } catch { /* model has no JSON mode */ }
|
||||
}
|
||||
return { spec: normalizeSpec(firstObject(raw)), raw: typeof raw === "string" ? raw : JSON.stringify(raw) };
|
||||
}
|
||||
|
||||
// Ask, check, and give the model one chance to fix what the validator found.
|
||||
// brain: "llm" writes the spec; "jev" / "clef" answer choice questions and decide.js assembles it. A decision that
|
||||
// fails validation, or whose weakest needed answer is under FLOOR, goes to the LLM: the cage with a door.
|
||||
export const BRAINS = ["llm", "jev", "clef"];
|
||||
async function specVia(env, dataset, prompt, prior, brain, floorAsked) {
|
||||
const info = describe(await rows(env), DATASETS[dataset].columns);
|
||||
const schema = { columns: info.columns };
|
||||
const t0 = Date.now();
|
||||
let handoff;
|
||||
if (brain !== "llm") {
|
||||
const model = brain === "jev" ? env.JEV_MODEL || "typesafe/jev" : env.CLEF_MODEL || "@cf/cloudflare/clef-flash";
|
||||
try {
|
||||
const d = await decide(env, model, info, prompt, prior && validateSpec(prior, schema).ok ? prior : undefined);
|
||||
// a caller may lower or raise the bar for itself (benchmarks; a page that prefers speed); it only changes cost
|
||||
const floor = Number.isFinite(floorAsked) && floorAsked >= 0 && floorAsked <= 1 ? floorAsked : Number(env.FLOOR || 0.5);
|
||||
const v = validateSpec(d.spec, schema);
|
||||
const confidences = Object.fromEntries(Object.entries(d.confidence).map(([k, c]) => [k, Math.round(c * 100) / 100]));
|
||||
if (v.ok && d.weakest >= floor)
|
||||
return { spec: d.spec, brain, model: d.model, ms: d.ms, confidence: d.weakest, confidences, alternatives: d.alternatives };
|
||||
handoff = { reason: v.ok ? `low confidence (${d.weakest} on ${d.weakestOn})` : "decision failed checks: " + v.errors[0],
|
||||
decided: d.spec, ms: d.ms, confidences };
|
||||
} catch (e) {
|
||||
handoff = { reason: "decision model error: " + String(e.message || e).slice(0, 160) };
|
||||
}
|
||||
}
|
||||
const out = await specFor(env, dataset, prompt, prior, info);
|
||||
return { ...out, brain: "llm", model: env.AI_MODEL, ms: Date.now() - t0, ...(handoff ? { handoff } : {}) };
|
||||
}
|
||||
|
||||
async function specFor(env, dataset, prompt, prior, known) {
|
||||
const info = known || describe(await rows(env), DATASETS[dataset].columns);
|
||||
const schema = { columns: info.columns };
|
||||
const messages = [{ role: "system", content: systemPrompt(dataset, info) }];
|
||||
if (prior && validateSpec(prior, schema).ok)
|
||||
messages.push({ role: "user", content: "Current chart spec (change it if the question builds on it): " + JSON.stringify(prior) });
|
||||
messages.push({ role: "user", content: prompt });
|
||||
let { spec, raw } = await model(env, messages), v = validateSpec(spec, schema);
|
||||
if (!v.ok) {
|
||||
messages.push({ role: "assistant", content: raw.slice(0, 2000) },
|
||||
{ role: "user", content: `That spec fails these checks: ${v.errors.join("; ")}. Answer with the corrected JSON object only.` });
|
||||
({ spec, raw } = await model(env, messages));
|
||||
v = validateSpec(spec, schema);
|
||||
}
|
||||
// On failure the model's own answer comes back too: it is shown, never run.
|
||||
return v.ok ? { spec } : { error: "couldn't turn that into a valid chart: " + v.errors.slice(0, 3).join("; "), model_said: raw.slice(0, 800) };
|
||||
}
|
||||
|
||||
const newId = () => [...crypto.getRandomValues(new Uint8Array(6))].map((b) => b.toString(36).padStart(2, "0")).join("").slice(0, 10);
|
||||
|
||||
async function api(req, env, url) {
|
||||
const path = url.pathname;
|
||||
if (path === "/api/health") return json({ ok: true, rev: env.REV || "", ai: !!env.AI });
|
||||
await setup(env);
|
||||
const dataset = url.searchParams.get("dataset") || "orders";
|
||||
if (req.method === "GET" && (path === "/api/schema" || path === "/api/data")) {
|
||||
if (!DATASETS[dataset]) return json({ error: "no such dataset" }, 404);
|
||||
const data = await rows(env);
|
||||
if (path === "/api/data") return json({ dataset, rows: data });
|
||||
const { columns, sample, aggregates } = describe(data, DATASETS[dataset].columns);
|
||||
return json({ dataset, title: DATASETS[dataset].title, schema: { columns }, sample, aggregates });
|
||||
}
|
||||
if (req.method === "POST" && (path === "/api/spec" || path === "/api/charts")) {
|
||||
let body;
|
||||
try { body = await req.json(); } catch { return json({ error: "send JSON" }, 400); }
|
||||
const ds = body?.dataset || "orders";
|
||||
if (!DATASETS[ds]) return json({ error: "no such dataset" }, 404);
|
||||
const prompt = String(body.prompt || "").trim().slice(0, MAX_PROMPT);
|
||||
if (path === "/api/spec") {
|
||||
if (!env.AI) return json({ error: "no AI here" }, 501);
|
||||
if (!prompt) return json({ error: "ask a question" }, 400);
|
||||
if (!(await spend(env, "ai", await who(req), 60, Number(env.AI_DAILY || 300)))) return json({ error: "over today's limit, try tomorrow" }, 429);
|
||||
const brain = BRAINS.includes(body.brain) ? body.brain : BRAINS.includes(env.BRAIN) ? env.BRAIN : "llm";
|
||||
const out = await specVia(env, ds, prompt, body.prior, brain, body.floor == null ? undefined : Number(body.floor));
|
||||
return json(out, out.spec ? 200 : 422);
|
||||
}
|
||||
const v = validateSpec(body.spec, { columns: DATASETS[ds].columns });
|
||||
if (!v.ok) return json({ error: "that spec doesn't validate: " + v.errors.join("; ") }, 400);
|
||||
if (!(await spend(env, "save", await who(req), 50, Number(env.SAVE_DAILY || 1000)))) return json({ error: "over today's limit" }, 429);
|
||||
const id = newId(), title = String(body.spec.title || prompt || "Chart").slice(0, 120);
|
||||
await env.DB.prepare("INSERT INTO saved_charts (id, created, dataset, prompt, title, spec) VALUES (?, ?, ?, ?, ?, ?)")
|
||||
.bind(id, new Date().toISOString(), ds, prompt, title, JSON.stringify(body.spec)).run();
|
||||
return json({ id, link: `${url.origin}/?c=${id}` }, 201);
|
||||
}
|
||||
if (req.method === "GET" && path === "/api/charts") {
|
||||
const list = (await env.DB.prepare("SELECT id, created, dataset, prompt, title FROM saved_charts ORDER BY created DESC LIMIT 20").all()).results;
|
||||
return json({ charts: list });
|
||||
}
|
||||
const one = path.match(/^\/api\/charts\/([a-z0-9]{4,16})$/);
|
||||
if (one && req.method === "GET") {
|
||||
const r = await env.DB.prepare("SELECT id, created, dataset, prompt, title, spec FROM saved_charts WHERE id = ?").bind(one[1]).first();
|
||||
return r ? json({ ...r, spec: JSON.parse(r.spec) }) : json({ error: "not found" }, 404);
|
||||
}
|
||||
return json({ error: "no such endpoint" }, 404);
|
||||
}
|
||||
|
||||
export const PAGE = `<!doctype html>
|
||||
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>AICharts</title>
|
||||
<meta name="description" content="Ask for a chart in plain words. The model writes a small checked spec, never code, and never sees your rows.">
|
||||
<style>
|
||||
:root{--bg:#faf9f6;--fg:#1d1d1b;--mute:#6b6a65;--line:#e3e1da;--card:#fff;--accent:#2f6fdf}
|
||||
@media (prefers-color-scheme:dark){:root{--bg:#161614;--fg:#ecebe6;--mute:#9b9a94;--line:#2c2b28;--card:#1f1f1c;--accent:#7aa5ff}}
|
||||
*{box-sizing:border-box}body{margin:0;background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,sans-serif}
|
||||
main{max-width:860px;margin:0 auto;padding:32px 16px 64px}h1{margin:0 0 4px;font-size:28px}p.lede{margin:0 0 20px;color:var(--mute)}
|
||||
.aic-ask{display:flex;gap:8px}.aic-ask input{flex:1;min-width:0;padding:10px 12px;border:1px solid var(--line);border-radius:8px;background:var(--card);color:var(--fg);font:inherit}
|
||||
button{padding:10px 14px;border:0;border-radius:8px;background:var(--accent);color:#fff;font:inherit;cursor:pointer}
|
||||
.aic-msg{min-height:24px;margin:10px 0;color:var(--mute)}.aic-canvas{background:var(--card);border:1px solid var(--line);border-radius:10px;padding:12px}
|
||||
.aic-save{margin-top:10px}.chips{display:flex;flex-wrap:wrap;gap:6px;margin:12px 0}.chips button{background:var(--card);color:var(--fg);border:1px solid var(--line);padding:6px 10px;font-size:14px}
|
||||
details{margin-top:16px}pre{background:var(--card);border:1px solid var(--line);border-radius:8px;padding:12px;overflow:auto;font-size:13px}
|
||||
h2{font-size:18px;margin:28px 0 8px}.row{display:flex;flex-wrap:wrap;gap:8px;align-items:center;justify-content:space-between}
|
||||
.brain{font-size:14px;color:var(--mute)}.brain select{font:inherit;padding:4px;border-radius:6px;border:1px solid var(--line);background:var(--card);color:var(--fg)}
|
||||
.meta{display:flex;flex-wrap:wrap;gap:6px;align-items:center;margin-top:10px;font-size:14px;color:var(--mute)}.meta button{padding:4px 10px;font-size:14px;background:var(--card);color:var(--fg);border:1px solid var(--line)}ul{padding-left:18px}a{color:var(--accent)}
|
||||
</style></head><body><main>
|
||||
<h1>AICharts</h1>
|
||||
<p class="lede">Ask for a chart in plain words. The model writes a small spec that is checked before anything draws, and never sees the rows: only column names, five sample rows and value ranges. Data: a sample tea shop, 24 months of orders.</p>
|
||||
<div class="row"><div class="chips" id="chips"></div><label class="brain">Brain <select id="brain"><option value="">default</option><option value="jev">Jev</option><option value="clef">Clef</option><option value="llm">LLM</option></select></label></div>
|
||||
<ai-chart dataset="orders"></ai-chart>
|
||||
<div class="meta" id="meta"></div>
|
||||
<details><summary>Spec</summary><pre id="spec">Ask something first.</pre></details>
|
||||
<h2>Saved charts</h2><ul id="saved"><li>None yet.</li></ul>
|
||||
<h2>Use it on your page</h2>
|
||||
<pre><script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.4/dist/chart.umd.min.js"></script>
|
||||
<script type="module" src="__ORIGIN__/aicharts.js"></script>
|
||||
<ai-chart endpoint="__ORIGIN__" dataset="orders"></ai-chart></pre>
|
||||
</main>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.4/dist/chart.umd.min.js"></script>
|
||||
<script type="module">
|
||||
import "/aicharts.js";
|
||||
const el = document.querySelector("ai-chart"), list = document.getElementById("saved");
|
||||
for (const q of ["Revenue by region", "Monthly revenue, web vs store", "Share of units by product", "Top 3 products in the West by revenue"]) {
|
||||
const b = document.createElement("button"); b.textContent = q; b.onclick = () => { el.querySelector("input").value = q; el.ask(q); };
|
||||
document.getElementById("chips").append(b);
|
||||
}
|
||||
const brainSel = document.getElementById("brain"), meta = document.getElementById("meta");
|
||||
brainSel.onchange = () => brainSel.value ? el.setAttribute("brain", brainSel.value) : el.removeAttribute("brain");
|
||||
el.addEventListener("spec", (e) => {
|
||||
const { spec, info = {} } = e.detail;
|
||||
document.getElementById("spec").textContent = JSON.stringify({ ...info, spec }, null, 2);
|
||||
const bits = [info.brain && "via " + info.brain + (info.handoff ? " (handed off: " + info.handoff.reason + ")" : ""),
|
||||
info.ms != null && info.ms + " ms", info.confidence != null && "confidence " + info.confidence].filter(Boolean);
|
||||
meta.replaceChildren(document.createTextNode(bits.join(" · ")));
|
||||
for (const alt of info.alternatives || []) {
|
||||
const b = document.createElement("button"); b.textContent = "or as " + alt.mark + " (" + Math.round(alt.p * 100) + "%)";
|
||||
b.onclick = () => el.render({ ...spec, mark: alt.mark }, e.detail.prompt).catch((err) => { meta.append(" " + err.message); });
|
||||
meta.append(b);
|
||||
}
|
||||
});
|
||||
el.onSave = async (d) => {
|
||||
const r = await fetch("/api/charts", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(d) });
|
||||
const out = await r.json(); if (!r.ok) throw new Error(out.error); history.replaceState(null, "", "?c=" + out.id); await refresh();
|
||||
};
|
||||
async function show(id) {
|
||||
const r = await fetch("/api/charts/" + id); if (!r.ok) return;
|
||||
const c = await r.json(); await el.render(c.spec, c.prompt); document.getElementById("spec").textContent = JSON.stringify(c.spec, null, 2);
|
||||
el.querySelector("input").value = c.prompt || "";
|
||||
}
|
||||
async function refresh() {
|
||||
const { charts } = await (await fetch("/api/charts")).json();
|
||||
list.replaceChildren(...(charts.length ? charts.map((c) => { const li = document.createElement("li"), a = document.createElement("a");
|
||||
a.href = "?c=" + c.id; a.textContent = c.title; a.onclick = (e) => { e.preventDefault(); history.replaceState(null, "", "?c=" + c.id); show(c.id); };
|
||||
li.append(a); return li; }) : [Object.assign(document.createElement("li"), { textContent: "None yet." })]));
|
||||
}
|
||||
const id = new URLSearchParams(location.search).get("c");
|
||||
customElements.whenDefined("ai-chart").then(() => { refresh(); if (id) show(id); });
|
||||
</script></body></html>`;
|
||||
|
||||
export function make(librarySource) {
|
||||
return {
|
||||
async fetch(req, env) {
|
||||
const url = new URL(req.url);
|
||||
if (req.method === "OPTIONS") return new Response(null, { status: 204, headers: CORS });
|
||||
try {
|
||||
if (url.pathname.startsWith("/api/")) return await api(req, env, url);
|
||||
} catch (e) {
|
||||
return json({ error: "something broke: " + String(e.message || e).slice(0, 160) }, 500);
|
||||
}
|
||||
if (url.pathname === "/aicharts.js")
|
||||
return new Response(librarySource, { headers: { "content-type": "text/javascript; charset=utf-8", "cache-control": "public, max-age=300", ...CORS } });
|
||||
if (url.pathname === "/" || url.pathname === "/index.html")
|
||||
return new Response(PAGE.replaceAll("__ORIGIN__", url.origin), { headers: { "content-type": "text/html; charset=utf-8" } });
|
||||
return new Response("not found", { status: 404 });
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -10,5 +10,9 @@
|
||||
"glm": {"family": "glm", "default": "z-ai-glm-5-3", "label": "Zhipu GLM via Venice",
|
||||
"aliases": {"flash": "z-ai-glm-5-3-flash"}},
|
||||
"gemini": {"family": "gemini", "default": "gemini-3-1-pro-preview", "label": "Google Gemini via Venice",
|
||||
"aliases": {"pro": "gemini-3-1-pro-preview", "flash": "gemini-3-8-flash"}}
|
||||
"aliases": {"pro": "gemini-3-1-pro-preview", "flash": "gemini-3-8-flash"}},
|
||||
"haiku": {"family": "claude-haiku", "default": "claude-haiku-5-5", "label": "Claude Haiku (Anthropic)", "provider": "anthropic", "price": [0.10, 0.50]},
|
||||
"sonnet": {"family": "claude-sonnet", "default": "claude-sonnet-5-5", "label": "Claude Sonnet (Anthropic)", "provider": "anthropic", "price": [2, 10]},
|
||||
"opus": {"family": "claude-opus", "default": "claude-opus-5-5", "label": "Claude Opus (Anthropic)", "provider": "anthropic", "price": [4, 20]},
|
||||
"fable": {"family": "claude-fable", "default": "claude-fable-5-1", "label": "Claude Fable (Anthropic)", "provider": "anthropic", "price": [10, 50]}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Seed a venture on demand: land a site built elsewhere on gh-pages at ventures/<slug>/, which Teahub pages serves at
|
||||
<slug>.yolopreneur.lol, and keep the venture's line in the ledger (ventures/ventures.json).
|
||||
|
||||
Run by .github/workflows/seed.yml (Actions -> seed -> Run workflow). Its inputs arrive in the environment:
|
||||
|
||||
SLUG the folder and the subdomain
|
||||
BUNDLE one or more https URLs, space-separated. Downloaded in order and joined, they make one .tgz whose
|
||||
top-level folder is <slug>/ (a big bundle can come as parts, e.g. Boring blob download links)
|
||||
SHA sha256 of the joined .tgz: nothing lands unless it matches
|
||||
NAME, ONE_LINER the ledger line; a venture already in the ledger keeps whatever is left empty
|
||||
|
||||
Nothing in the bundle runs. It is checked (paths, file types, size, an index.html) and copied over ventures/<slug>/,
|
||||
replacing what was there, in one commit pushed to gh-pages by the Actions identity.
|
||||
|
||||
python3 seed.py PAGES_DIR # PAGES_DIR: a checkout of gh-pages
|
||||
"""
|
||||
import datetime as dt
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
MAX_BYTES = 64 * 1024 * 1024
|
||||
SLUG = re.compile(r"[a-z0-9](?:[a-z0-9-]{0,40}[a-z0-9])?")
|
||||
RESERVED = {"index.html", "ventures.json", "backlog.json", "shots", "assets", "www"}
|
||||
SITE = "https://{slug}.yolopreneur.lol/"
|
||||
|
||||
|
||||
class SeedError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def fetch(urls):
|
||||
"""the bundle: every URL in order, joined"""
|
||||
data = bytearray()
|
||||
for u in urls:
|
||||
if not u.startswith("https://"):
|
||||
raise SeedError(f"not an https URL: {u[:60]}")
|
||||
with urllib.request.urlopen(u, timeout=120) as r:
|
||||
data += r.read(MAX_BYTES + 1 - len(data))
|
||||
if len(data) > MAX_BYTES:
|
||||
raise SeedError(f"bundle is over {MAX_BYTES // 2**20} MB")
|
||||
return bytes(data)
|
||||
|
||||
|
||||
def unpack(data, slug, into):
|
||||
"""write the bundle's <slug>/ tree under `into`, refusing anything that isn't a plain file or folder inside it"""
|
||||
with tarfile.open(fileobj=io.BytesIO(data), mode="r:gz") as tf:
|
||||
members = tf.getmembers()
|
||||
for m in members:
|
||||
parts = Path(m.name).parts
|
||||
if m.name.startswith("/") or ".." in parts:
|
||||
raise SeedError(f"unsafe path in bundle: {m.name}")
|
||||
if not (m.isfile() or m.isdir()):
|
||||
raise SeedError(f"not a plain file or folder: {m.name}")
|
||||
if not parts or parts[0] != slug:
|
||||
raise SeedError(f"everything in the bundle must sit under {slug}/: {m.name}")
|
||||
for m in members:
|
||||
target = into / m.name
|
||||
if m.isdir():
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
else:
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(tf.extractfile(m).read())
|
||||
if not (into / slug / "index.html").is_file():
|
||||
raise SeedError(f"the bundle has no {slug}/index.html")
|
||||
|
||||
|
||||
def ledger_line(data, slug, name, one_liner, today):
|
||||
"""the venture's line in the ledger, added (at the top) or refreshed; raises before anything is written"""
|
||||
vs = data.setdefault("ventures", [])
|
||||
v = next((v for v in vs if v.get("slug") == slug), None)
|
||||
if v is None:
|
||||
if not name or not one_liner:
|
||||
raise SeedError("a new venture needs a name and a one-liner for the ledger")
|
||||
v = {"slug": slug, "name": name, "one_liner": one_liner, "url": SITE.format(slug=slug),
|
||||
"pages": SITE.format(slug=slug), "launched": today, "signups": 0, "source": "yolopreneur"}
|
||||
vs.insert(0, v)
|
||||
else:
|
||||
if name:
|
||||
v["name"] = name
|
||||
if one_liner:
|
||||
v["one_liner"] = one_liner
|
||||
return v
|
||||
|
||||
|
||||
def land(pages, slug, data, want, name="", one_liner="", today=None):
|
||||
"""check the bundle and put it at ventures/<slug>/ in the gh-pages checkout `pages`; returns the bundle's sha256.
|
||||
Every check runs before anything is written, so a refused bundle leaves the checkout as it was."""
|
||||
if not SLUG.fullmatch(slug or "") or slug in RESERVED:
|
||||
raise SeedError(f"bad slug: {slug!r}")
|
||||
if not re.fullmatch(r"[0-9a-f]{64}", (want or "").lower()):
|
||||
raise SeedError("sha256 must be 64 hex characters")
|
||||
got = hashlib.sha256(data).hexdigest()
|
||||
if got != want.lower():
|
||||
raise SeedError(f"sha256 mismatch: the bundle is {got}")
|
||||
path = pages / "ventures" / "ventures.json"
|
||||
book = json.loads(path.read_text(encoding="utf-8")) if path.exists() else {"ventures": []}
|
||||
line = ledger_line(book, slug, name, one_liner, today or dt.date.today().isoformat())
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
unpack(data, slug, Path(tmp))
|
||||
dest = pages / "ventures" / slug
|
||||
shutil.rmtree(dest, ignore_errors=True)
|
||||
shutil.copytree(Path(tmp) / slug, dest)
|
||||
if (dest / "shots" / "desktop.png").is_file():
|
||||
line["shot"] = f"{slug}/shots/desktop.png" # the card on the yolopreneur.lol index
|
||||
path.write_text(json.dumps(book, indent=1), encoding="utf-8") # the ledger's own layout: indent 1, ASCII
|
||||
return got
|
||||
|
||||
|
||||
def commit(pages, slug, sha):
|
||||
git = ["git", "-C", str(pages), "-c", "user.name=github-actions[bot]",
|
||||
"-c", "user.email=github-actions[bot]@users.noreply.teahub.xyz"]
|
||||
subprocess.run(git + ["add", "--all", f"ventures/{slug}", "ventures/ventures.json"], check=True)
|
||||
if subprocess.run(git + ["diff", "--cached", "--quiet"]).returncode == 0:
|
||||
print("nothing changed: ventures/" + slug + " is already this bundle")
|
||||
return False
|
||||
subprocess.run(git + ["commit", "-q", "-m", f"ventures: seed {slug} (bundle {sha[:12]})"], check=True)
|
||||
if subprocess.run(git + ["push", "-q", "origin", "HEAD:gh-pages"]).returncode != 0:
|
||||
# gh-pages moved under us (the venture loop writes there too): replay this one commit on top, push again
|
||||
subprocess.run(git + ["pull", "-q", "--rebase", "origin", "gh-pages"], check=True)
|
||||
subprocess.run(git + ["push", "-q", "origin", "HEAD:gh-pages"], check=True)
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
pages = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
|
||||
slug = os.environ.get("SLUG", "").strip()
|
||||
try:
|
||||
data = fetch(os.environ.get("BUNDLE", "").split())
|
||||
sha = land(pages, slug, data, os.environ.get("SHA", "").strip(), os.environ.get("NAME", "").strip(),
|
||||
os.environ.get("ONE_LINER", "").strip())
|
||||
except (SeedError, OSError, tarfile.TarError) as e:
|
||||
print(f"::error::seed {slug}: {e}")
|
||||
sys.exit(1)
|
||||
pushed = commit(pages, slug, sha)
|
||||
print(f"{'landed' if pushed else 'unchanged'}: ventures/{slug}/ ({len(data)} bytes, sha256 {sha[:12]}) -> "
|
||||
+ SITE.format(slug=slug))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for seed.py: a bundle lands only when it is what it says it is, and the ledger keeps its layout."""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import io
|
||||
import json
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
spec = importlib.util.spec_from_file_location("seed", Path(__file__).with_name("seed.py"))
|
||||
seed = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(seed)
|
||||
|
||||
LEDGER = {"ventures": [{"slug": "mdecks", "name": ".mdeck", "one_liner": "Slides in plain text — fast.",
|
||||
"launched": "2026-10-12", "signups": 0}], "refreshed": "2026-10-14"}
|
||||
|
||||
|
||||
def tgz(files, links=()):
|
||||
buf = io.BytesIO()
|
||||
with tarfile.open(fileobj=buf, mode="w:gz") as tf:
|
||||
for name, body in files.items():
|
||||
ti = tarfile.TarInfo(name)
|
||||
ti.size = len(body)
|
||||
tf.addfile(ti, io.BytesIO(body))
|
||||
for name, target in links:
|
||||
ti = tarfile.TarInfo(name)
|
||||
ti.type, ti.linkname = tarfile.SYMTYPE, target
|
||||
tf.addfile(ti)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def pages_dir():
|
||||
d = Path(tempfile.mkdtemp())
|
||||
(d / "ventures").mkdir()
|
||||
(d / "ventures" / "ventures.json").write_text(json.dumps(LEDGER, indent=1), encoding="utf-8")
|
||||
return d
|
||||
|
||||
|
||||
def refused(fn, *a, **kw):
|
||||
try:
|
||||
fn(*a, **kw)
|
||||
except seed.SeedError as e:
|
||||
return str(e)
|
||||
raise AssertionError("expected SeedError")
|
||||
|
||||
|
||||
def sha(b):
|
||||
return hashlib.sha256(b).hexdigest()
|
||||
|
||||
|
||||
def test_lands_and_lists():
|
||||
p = pages_dir()
|
||||
b = tgz({"hh/index.html": b"<!doctype html>hi", "hh/plates/a.svg": b"<svg/>", "hh/shots/desktop.png": b"png"})
|
||||
assert seed.land(p, "hh", b, sha(b), "House Hand", "A drawing hand.", today="2026-10-15") == sha(b)
|
||||
assert (p / "ventures/hh/plates/a.svg").read_bytes() == b"<svg/>"
|
||||
text = (p / "ventures/ventures.json").read_text()
|
||||
assert text == json.dumps(json.loads(text), indent=1) # same layout as the file it came from
|
||||
assert "\\u2014" in text # ASCII escapes kept, like the original
|
||||
first = json.loads(text)["ventures"][0]
|
||||
assert first["slug"] == "hh" and first["launched"] == "2026-10-15" and first["shot"] == "hh/shots/desktop.png"
|
||||
assert first["url"] == "https://hh.yolopreneur.lol/" and json.loads(text)["refreshed"] == "2026-10-14"
|
||||
|
||||
|
||||
def test_replaces_and_keeps_the_ledger_line():
|
||||
p = pages_dir()
|
||||
b1 = tgz({"hh/index.html": b"one", "hh/old.txt": b"stale"})
|
||||
seed.land(p, "hh", b1, sha(b1), "House Hand", "First line.", today="2026-10-15")
|
||||
b2 = tgz({"hh/index.html": b"two"})
|
||||
seed.land(p, "hh", b2, sha(b2), today="2026-10-20") # no name or line given: keep them
|
||||
assert not (p / "ventures/hh/old.txt").exists() and (p / "ventures/hh/index.html").read_bytes() == b"two"
|
||||
vs = [v for v in json.loads((p / "ventures/ventures.json").read_text())["ventures"] if v["slug"] == "hh"]
|
||||
assert len(vs) == 1 and vs[0]["one_liner"] == "First line." and vs[0]["launched"] == "2026-10-15"
|
||||
|
||||
|
||||
def test_refuses():
|
||||
p = pages_dir()
|
||||
good = tgz({"hh/index.html": b"x"})
|
||||
assert "mismatch" in refused(seed.land, p, "hh", good, "0" * 64, "n", "o")
|
||||
assert "bad slug" in refused(seed.land, p, "../etc", good, sha(good), "n", "o")
|
||||
assert "bad slug" in refused(seed.land, p, "ventures.json", good, sha(good), "n", "o")
|
||||
for bad, why in ((tgz({"hh/../../x": b"x", "hh/index.html": b"x"}), "unsafe"),
|
||||
(tgz({"hh/index.html": b"x"}, links=[("hh/link", "/etc/passwd")]), "plain file"),
|
||||
(tgz({"other/index.html": b"x"}), "under hh/"),
|
||||
(tgz({"hh/readme.txt": b"x"}), "no hh/index.html")):
|
||||
assert why in refused(seed.land, p, "hh", bad, sha(bad), "n", "o"), why
|
||||
assert "needs a name" in refused(seed.land, p, "hh", good, sha(good))
|
||||
assert not (p / "ventures/hh").exists() # nothing half-landed
|
||||
assert "https" in refused(seed.fetch, ["http://example.com/x.tgz"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
tests = [v for k, v in sorted(globals().items()) if k.startswith("test_")]
|
||||
for t in tests:
|
||||
t()
|
||||
print("ok", t.__name__)
|
||||
print(f"{len(tests)} seed tests passed")
|
||||
sys.exit(0)
|
||||
+1199
-43
File diff suppressed because it is too large.
Load diff
+560
-120
File diff suppressed because it is too large.
Load diff
+1697
-112
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,34 @@
|
||||
name: astra
|
||||
# The slow loop: once a day, one planning call (TIME_ASTRA_SEAT, default `opus`) reads the board and the ledger and
|
||||
# labels a few cards `ready` for the heartbeat. It writes its reasoning to the "Astra log" issue; reply there to steer it.
|
||||
# Guardrails are code: TIME_ASTRA_MAX_READY (2) / TIME_ASTRA_MAX_NEW (2) per day, nothing while an `andon` is open,
|
||||
# and the same spend caps as every other call. Off until TIME_ASTRA=on.
|
||||
on:
|
||||
schedule:
|
||||
- cron: "13 13 * * *"
|
||||
workflow_dispatch:
|
||||
concurrency:
|
||||
group: astra
|
||||
cancel-in-progress: false
|
||||
jobs:
|
||||
plan:
|
||||
if: vars.TIME_ASTRA == 'on'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
sparse-checkout: .github/time
|
||||
persist-credentials: false
|
||||
- run: python3 .github/time/astra.py tick
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
TIME_TOKEN: ${{ secrets.TIME_TOKEN }} # repo secret: Astra's issues and labels go out as the `time` bot
|
||||
VENICE_API_KEY: ${{ secrets.VENICE_API_KEY }}
|
||||
TIME_ASTRA_SEAT: ${{ vars.TIME_ASTRA_SEAT }}
|
||||
TIME_ASTRA_MAX_READY: ${{ vars.TIME_ASTRA_MAX_READY }}
|
||||
TIME_ASTRA_MAX_NEW: ${{ vars.TIME_ASTRA_MAX_NEW }}
|
||||
TIME_SPINE: ${{ vars.TIME_SPINE }}
|
||||
TIME_DAILY_USD: ${{ vars.TIME_DAILY_USD }}
|
||||
TIME_MONTHLY_USD: ${{ vars.TIME_MONTHLY_USD }}
|
||||
@@ -4,6 +4,8 @@ on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
paths-ignore: ["time/ledger/**"] # Time appends its own spend ledger every run; that is not a code change
|
||||
workflow_dispatch: # Time can run CI on a worker's PR branch by hand
|
||||
permissions:
|
||||
contents: read
|
||||
jobs:
|
||||
@@ -12,4 +14,8 @@ jobs:
|
||||
timeout-minutes: 3
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: actions/setup-node@v4 # node 22+ has node:sqlite, which the passkey test runs on
|
||||
with:
|
||||
node-version: 22
|
||||
- run: python3 .github/time/test_time.py
|
||||
- run: python3 .github/time/test_seed.py
|
||||
@@ -1,5 +1,5 @@
|
||||
name: doctor
|
||||
# Manual check of Time's setup: label, settings, keys (never printed). Result in the run summary.
|
||||
# Manual check of Time's setup: label, settings, keys (never printed), today's spend. Result in the run summary.
|
||||
on: workflow_dispatch
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -18,8 +18,14 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
|
||||
VENICE_API_KEY: ${{ secrets.VENICE_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
TIME_VENTURE_DOMAIN: ${{ vars.TIME_VENTURE_DOMAIN }}
|
||||
TIME_MODE: ${{ vars.TIME_MODE }}
|
||||
TIME_DAILY_CAP: ${{ vars.TIME_DAILY_CAP }}
|
||||
TIME_DAILY_USD: ${{ vars.TIME_DAILY_USD }}
|
||||
TIME_MONTHLY_USD: ${{ vars.TIME_MONTHLY_USD }}
|
||||
TIME_REVIEW: ${{ vars.TIME_REVIEW }}
|
||||
TIME_WORKER: ${{ vars.TIME_WORKER }}
|
||||
TIME_AUTOMERGE: ${{ vars.TIME_AUTOMERGE }}
|
||||
@@ -11,21 +11,16 @@ on:
|
||||
types: [submitted]
|
||||
issue_comment:
|
||||
types: [created]
|
||||
permissions:
|
||||
contents: write # squash-merge approved, green PRs
|
||||
issues: write
|
||||
pull-requests: write
|
||||
checks: read
|
||||
statuses: write # a Venice worker's test run is recorded as the PR's check
|
||||
# (GitHub-style `permissions:` are ignored by Gitea; the Actions token has write access to this repo)
|
||||
concurrency:
|
||||
group: heartbeat
|
||||
cancel-in-progress: false
|
||||
jobs:
|
||||
tick:
|
||||
# issues: only `ready`/`spec` labels. comments: only on PRs, only a reviewer verdict (Approve/Object).
|
||||
# issues: only `ready`/`spec` labels. comments: only on PRs, only a reviewer verdict (Approve/Object), never Time's own.
|
||||
if: >-
|
||||
(github.event_name != 'issues' || contains(fromJSON('["ready","spec"]'), github.event.label.name)) &&
|
||||
(github.event_name != 'issue_comment' || (github.event.issue.pull_request &&
|
||||
(github.event_name != 'issue_comment' || (github.event.issue.pull_request && github.event.comment.user.login != 'gitea-actions' &&
|
||||
(startsWith(github.event.comment.body, '**Approve') || startsWith(github.event.comment.body, 'Approve') ||
|
||||
startsWith(github.event.comment.body, '**Object') || startsWith(github.event.comment.body, 'Object'))))
|
||||
runs-on: ubuntu-latest
|
||||
@@ -34,13 +29,20 @@ jobs:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
sparse-checkout: .github/time
|
||||
persist-credentials: false # the worker's shell must not find the token in .git/config
|
||||
- run: python3 .github/time/time.py heartbeat
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
|
||||
VENICE_API_KEY: ${{ secrets.VENICE_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # user-level secret on teahubxyz (docs/decisions.md on the spine)
|
||||
TIME_TOKEN: ${{ secrets.TIME_TOKEN }} # repo secret: the `time` bot opens and merges PRs (Gitea denies that to the Actions identity)
|
||||
TIME_TRUSTED: ${{ vars.TIME_TRUSTED }} # org owners etc.; collaborators are trusted without being listed
|
||||
TIME_RUN_IMAGE: node:22-bookworm # the worker's shell runs in a throwaway container, not next to these secrets
|
||||
TIME_MODE: ${{ vars.TIME_MODE }}
|
||||
TIME_DAILY_CAP: ${{ vars.TIME_DAILY_CAP }}
|
||||
TIME_DAILY_USD: ${{ vars.TIME_DAILY_USD }} # spend governor: dollars per day (default 5) …
|
||||
TIME_MONTHLY_USD: ${{ vars.TIME_MONTHLY_USD }} # … and per month (default 100); ledger in time/ledger/
|
||||
TIME_CURSOR_MODEL: ${{ vars.TIME_CURSOR_MODEL }}
|
||||
TIME_REVIEW: ${{ vars.TIME_REVIEW }}
|
||||
TIME_STUCK_HOURS: ${{ vars.TIME_STUCK_HOURS }}
|
||||
@@ -51,3 +53,4 @@ jobs:
|
||||
TIME_PLANNER: ${{ vars.TIME_PLANNER }}
|
||||
TIME_WORKER: ${{ vars.TIME_WORKER }}
|
||||
TIME_WORKER_STEPS: ${{ vars.TIME_WORKER_STEPS }}
|
||||
TIME_WORKER_MAX_TOKENS: ${{ vars.TIME_WORKER_MAX_TOKENS }}
|
||||
@@ -0,0 +1,38 @@
|
||||
name: labs-aicharts
|
||||
# Deploy the AICharts demo Worker (aicharts.<venture domain>, one D1, Workers AI), then smoke-test it. Manual only.
|
||||
on: workflow_dispatch
|
||||
permissions:
|
||||
contents: read
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
sparse-checkout: .github/time
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
- run: node .github/time/labs/aicharts/test_aicharts.mjs
|
||||
- id: deploy
|
||||
run: |
|
||||
set -o pipefail
|
||||
python3 .github/time/labs/aicharts/deploy.py | tee out.json
|
||||
echo "url=$(python3 -c 'import json; print(json.load(open("out.json"))["url"])')" >> "$GITHUB_OUTPUT"
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
TIME_VENTURE_DOMAIN: ${{ vars.TIME_VENTURE_DOMAIN }}
|
||||
- name: smoke
|
||||
env:
|
||||
u: ${{ steps.deploy.outputs.url }} # through env, never interpolated into the script
|
||||
run: |
|
||||
set -o pipefail
|
||||
for i in $(seq 1 18); do curl -fsS "${u}api/health" && break; sleep 10; done
|
||||
curl -fsS "${u}api/schema?dataset=orders" | python3 -c 'import json,sys; d=json.load(sys.stdin); assert len(d["sample"])==5, d; print("schema ok")'
|
||||
curl -fsS -X POST "${u}api/spec" -H 'content-type: application/json' \
|
||||
-d '{"dataset":"orders","prompt":"total revenue by region"}' | tee spec.json
|
||||
echo
|
||||
echo "### AICharts: ${u}" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo '```json' >> "$GITHUB_STEP_SUMMARY"; cat spec.json >> "$GITHUB_STEP_SUMMARY"; echo '```' >> "$GITHUB_STEP_SUMMARY"
|
||||
@@ -14,12 +14,13 @@ concurrency:
|
||||
cancel-in-progress: false
|
||||
jobs:
|
||||
reply:
|
||||
# cheap filter before a runner spins up: a mention, not from a bot, and only from people with a say in this repo
|
||||
# (public repo: strangers can comment, but must not be able to spend the seats' model credits)
|
||||
# cheap filter before a runner spins up: a mention, and not Time's own comment (Gitea fires events for the Actions
|
||||
# token's writes). Who may spend the seats' credits is checked in time.py (collaborator permission); Gitea has no
|
||||
# author_association in the event.
|
||||
if: >-
|
||||
contains(github.event.comment.body || github.event.issue.body, '@') &&
|
||||
(github.event.comment.user.type || github.event.issue.user.type) != 'Bot' &&
|
||||
contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association || github.event.issue.author_association)
|
||||
(github.event.comment.user.login || github.event.issue.user.login) != 'gitea-actions' &&
|
||||
(github.event.comment.user.login || github.event.issue.user.login) != 'time'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 8
|
||||
steps:
|
||||
@@ -30,3 +31,7 @@ jobs:
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VENICE_API_KEY: ${{ secrets.VENICE_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
TIME_TRUSTED: ${{ vars.TIME_TRUSTED }} # org owners etc.; collaborators are trusted without being listed
|
||||
TIME_DAILY_USD: ${{ vars.TIME_DAILY_USD }}
|
||||
TIME_MONTHLY_USD: ${{ vars.TIME_MONTHLY_USD }}
|
||||
@@ -0,0 +1,59 @@
|
||||
name: seed
|
||||
# Seed a venture on demand: land a site that was built somewhere else (by hand, on the Mac, by another agent) on
|
||||
# yolopreneur.lol. Actions -> seed -> Run workflow, with the inputs below.
|
||||
# The bundle is a .tgz whose top-level folder is the slug. .github/time/seed.py checks it against its sha256, copies
|
||||
# it over ventures/<slug>/ on gh-pages in one commit, adds or refreshes the venture's line in ventures/ventures.json,
|
||||
# and the last step asks Teahub pages to pull, so <slug>.yolopreneur.lol serves it within the minute.
|
||||
# Nothing in the bundle runs: it is checked (plain files and folders under <slug>/, an index.html) and copied.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
slug:
|
||||
description: "Folder and subdomain: ventures/<slug>/ is served at <slug>.yolopreneur.lol"
|
||||
required: true
|
||||
type: string
|
||||
bundle:
|
||||
description: "https URL of the .tgz, or of its parts, space-separated and in order"
|
||||
required: true
|
||||
type: string
|
||||
sha256:
|
||||
description: "sha256 of the whole .tgz"
|
||||
required: true
|
||||
type: string
|
||||
name:
|
||||
description: "Name in the ledger (needed for a new venture)"
|
||||
required: false
|
||||
type: string
|
||||
one_liner:
|
||||
description: "One line in the ledger (needed for a new venture)"
|
||||
required: false
|
||||
type: string
|
||||
permissions:
|
||||
contents: write
|
||||
concurrency:
|
||||
group: venture # the venture loop writes gh-pages too: take turns
|
||||
cancel-in-progress: false
|
||||
jobs:
|
||||
seed:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
sparse-checkout: .github/time
|
||||
path: engine
|
||||
persist-credentials: false
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
ref: gh-pages
|
||||
path: pages
|
||||
fetch-depth: 1
|
||||
- run: python3 engine/.github/time/seed.py pages
|
||||
env: # inputs reach the script through the environment, never interpolated into it
|
||||
SLUG: ${{ inputs.slug }}
|
||||
BUNDLE: ${{ inputs.bundle }}
|
||||
SHA: ${{ inputs.sha256 }}
|
||||
NAME: ${{ inputs.name }}
|
||||
ONE_LINER: ${{ inputs.one_liner }}
|
||||
- name: tell Teahub pages to pull
|
||||
run: curl -fsS -X POST "https://yolopreneur.lol/_hook/${GITHUB_REPOSITORY}" || echo "pages pulls by itself within five minutes"
|
||||
@@ -24,12 +24,12 @@ jobs:
|
||||
- name: copy engine, open PR if changed
|
||||
run: |
|
||||
set -e
|
||||
for f in .github/time/time.py .github/time/venture.py .github/time/test_time.py .github/time/seats.json .claude/skills/time/SKILL.md; do
|
||||
for f in .github/time/time.py .github/time/venture.py .github/time/cloudflare.py .github/time/cloudflare/worker.js .github/time/cloudflare/time.js .github/time/cloudflare/test_worker.mjs .github/time/cloudflare/test_auth.mjs .github/time/test_time.py .github/time/seats.json .github/ISSUE_TEMPLATE/early-access.yml .claude/skills/time/SKILL.md; do
|
||||
mkdir -p "$(dirname "$f")"; cp "/tmp/up/$f" "$f"
|
||||
done
|
||||
python3 .github/time/test_time.py > /dev/null
|
||||
wf=$(for f in /tmp/up/.github/workflows/*.yml; do b=.github/workflows/$(basename "$f"); cmp -s "$f" "$b" || echo "- \`$b\`"; done)
|
||||
if git diff --quiet; then
|
||||
if git diff --quiet && [ -z "$(git status --porcelain .github/time)" ]; then
|
||||
echo "Time engine is up to date." | tee -a "$GITHUB_STEP_SUMMARY"
|
||||
[ -n "$wf" ] && printf 'Workflow files differ upstream, copy by hand:\n%s\n' "$wf" | tee -a "$GITHUB_STEP_SUMMARY"
|
||||
exit 0
|
||||
@@ -37,6 +37,7 @@ jobs:
|
||||
sha=$(git -C /tmp/up rev-parse --short HEAD)
|
||||
branch="time-update-$sha"
|
||||
git switch -c "$branch"
|
||||
git add .github/time .github/ISSUE_TEMPLATE .claude/skills/time
|
||||
git -c user.name=time -c user.email=time@users.noreply.github.com commit -qam "Update Time to $UPSTREAM@$sha"
|
||||
git push -q origin "$branch"
|
||||
body="Fresh Time engine from \`$UPSTREAM@$CHANNEL\` ($sha). Offline tests pass."
|
||||
|
||||
@@ -1,31 +1,62 @@
|
||||
name: venture
|
||||
# The venture loop: one stage per tick. Daily kickoff plus a nudge every 20 minutes while a card is open
|
||||
# (cheap: a tick with no open card and TIME_VENTURE!=on exits at once). Off until TIME_VENTURE=on.
|
||||
# The venture loop: trending repos -> pitches -> board -> launch plan -> launch page on Pages -> 7 days of signups.
|
||||
# Every 20 minutes it advances whatever is due (cheap when nothing is); a new venture card starts once a day,
|
||||
# or right away: Actions -> venture -> Run workflow -> tick "Start a new venture now" (gh workflow run venture.yml -f start=true).
|
||||
# Batches: TIME_VENTURE_BATCH=3 (or Run workflow -> batch) runs k ventures at once in one milestone.
|
||||
# Off until TIME_VENTURE=on. Signups come in through .github/ISSUE_TEMPLATE/early-access.yml.
|
||||
on:
|
||||
schedule:
|
||||
- cron: "*/20 * * * *"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
start:
|
||||
description: "Start a new venture now, even if today's has already run"
|
||||
type: boolean
|
||||
default: false
|
||||
batch:
|
||||
description: "Start a venture batch now: one scout, the board's top k, launched together (k = TIME_VENTURE_BATCH, or 3)"
|
||||
type: boolean
|
||||
default: false
|
||||
permissions:
|
||||
contents: write
|
||||
issues: write
|
||||
pages: write
|
||||
statuses: write
|
||||
concurrency:
|
||||
group: venture
|
||||
cancel-in-progress: false
|
||||
jobs:
|
||||
tick:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
with:
|
||||
sparse-checkout: .github/time
|
||||
persist-credentials: false # the worker's shell must not find the token in .git/config
|
||||
- run: python3 .github/time/venture.py tick
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VENICE_API_KEY: ${{ secrets.VENICE_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
TIME_TOKEN: ${{ secrets.TIME_TOKEN }} # the `time` bot: opens and merges PRs (Gitea denies that to the Actions identity)
|
||||
TIME_RUN_IMAGE: node:22-bookworm # the worker's shell runs in a throwaway container, not next to these secrets
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} # optional: a backend per venture (cloudflare.py)
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
TIME_DAILY_USD: ${{ vars.TIME_DAILY_USD }}
|
||||
TIME_MONTHLY_USD: ${{ vars.TIME_MONTHLY_USD }}
|
||||
TIME_VENTURE_DOMAIN: ${{ vars.TIME_VENTURE_DOMAIN }}
|
||||
TIME_VENTURE_AI_MODEL: ${{ vars.TIME_VENTURE_AI_MODEL }}
|
||||
TIME_VENTURE: ${{ vars.TIME_VENTURE }}
|
||||
TIME_VENTURE_START: ${{ inputs.start && '1' || '' }}
|
||||
TIME_VENTURE_BATCH_START: ${{ inputs.batch && '1' || '' }}
|
||||
TIME_VENTURE_BATCH: ${{ vars.TIME_VENTURE_BATCH }} # k; set, a batch replaces the daily single card
|
||||
TIME_VENTURE_BATCH_HOLD_HOURS: ${{ vars.TIME_VENTURE_BATCH_HOLD_HOURS }} # launch-together hold (default 6)
|
||||
TIME_VENTURE_HEAVY: ${{ vars.TIME_VENTURE_HEAVY }} # worker runs at once per tick (default 3)
|
||||
TIME_VENTURE_PITCH: ${{ vars.TIME_VENTURE_PITCH }}
|
||||
TIME_VENTURE_BOARD: ${{ vars.TIME_VENTURE_BOARD }}
|
||||
TIME_VENTURE_CHAIR: ${{ vars.TIME_VENTURE_CHAIR }}
|
||||
TIME_VENTURE_MEASURE_DAYS: ${{ vars.TIME_VENTURE_MEASURE_DAYS }}
|
||||
TIME_WORKER: ${{ vars.TIME_WORKER }}
|
||||
TIME_WORKER_STEPS: ${{ vars.TIME_WORKER_STEPS }}
|
||||
TIME_WORKER_MAX_TOKENS: ${{ vars.TIME_WORKER_MAX_TOKENS }}
|
||||
@@ -2,5 +2,34 @@
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Security
|
||||
- The worker's own commit status (`time/worker-run`) can fail or delay a merge but never earns one: with no other check on the PR head, Time dispatches the project's CI workflow on the PR branch (`TIME_CI_WORKFLOW`, default `ci.yml`, which must accept `workflow_dispatch`; GitHub skips `pull_request` runs for pushes made with the Actions token) and merges on that. `off` keeps the old "no checks" stop.
|
||||
- `.github/` is protected from auto-merge whatever `TIME_PROTECTED` says.
|
||||
- `labs-aicharts.yml` passes the deploy URL through `env` instead of interpolating it into the shell.
|
||||
- Time's state comments (`<!-- time {...} -->` beats) count only when written by `github-actions[bot]`; a pasted marker from anyone else is ignored. A stranger on a public repo could otherwise steer a card to merge a chosen PR or build a chosen page.
|
||||
- `/time retry` and "Approve"/"Object" comments count only from the owner, members, collaborators and Time itself; any other bot is a stranger too.
|
||||
- The protected-path check reads every file of a PR (not the first 100) and counts a rename under both names.
|
||||
- The browser pass clones the Pages branch, then drops `.git` (the clone URL carried the token), serves only the venture's own folder, and runs headless Chrome with no network except localhost, so a model-written page can't read or send anything.
|
||||
- The worker's `run` tool executes inside a throwaway container (`docker run`, the checkout mounted; the workflows set `TIME_RUN_IMAGE: node:22-bookworm`), so a command can't read the orchestrator's environment through `/proc`. Unset, or without a docker daemon, it is the host shell as before.
|
||||
- A venture build writes only inside its own `ventures/<slug>/` folder on the Pages branch, which also carries the studio's site and every other venture: a fix round once wrote its page over the site's front page.
|
||||
- The Venice worker's shell no longer sees secrets: its environment drops tokens and keys, its checkout keeps no token in `.git/config`, and the workflows check out without persisted credentials.
|
||||
|
||||
### Changed
|
||||
- The second fix round escalates to Kimi's code model (`venice:kimi code`) instead of grok 4.7: still a different model from the first round, at a fraction of the cost. `TIME_VENTURE_ESCALATE` overrides it.
|
||||
|
||||
### Added
|
||||
- A batch's children start in the tick that spawned them (GitHub lists a card made a moment ago only later, so they used to wait a tick).
|
||||
- Build briefs keep claims to what the code does: no promised features or schedules, and the trending repos are the wave a venture rides, never something it "builds on". Fix rounds take a failing claim off the page rather than building a feature in a hurry. In the first live batch, every fix round was a claim the code didn't back.
|
||||
- Venture batches (multi-ceremony): one batch card scouts once, the seats pitch a field (rounds until there are 2k pitches), and the board ranks the whole field; the top k, each wanted by two seats, become venture cards in the batch's own milestone. They run the ordinary loop side by side and launch together: a child that passes its checks waits at `ready` until every sibling is settled or the hold (`TIME_VENTURE_BATCH_HOLD_HOURS`, 6) runs out; stragglers then launch alone. No child left behind: a held child gets one batch retry, and held again it is counted dead with its reason. A retro on the batch card compares the cohort's signups against each plan's lines, then the card and milestone close. Set `TIME_VENTURE_BATCH=3` to make batches the daily default, or Actions → venture → Run workflow → "batch".
|
||||
- Parallel builds: a tick runs up to `TIME_VENTURE_HEAVY` (default 3) worker runs at once, one per card, so a batch builds in one tick instead of three. Each build keeps its own folder lock, `ventures.json` is updated under a lock with a fresh read, and a Pages write that loses the race for the branch head (409) is retried. `TIME_VENTURE_HEAVY=1` is the old one-at-a-time.
|
||||
- Fix rounds yield the worker runs to cards still due their first build, so a page that keeps failing can't starve the others.
|
||||
- Accounts for ventures, with passkeys: each venture's backend can sign people up and in (Face ID, Touch ID, Windows Hello, a phone), keep a 30-day session, and list what each person saved. No passwords, no email, nothing to configure; accounts live in the venture's own database on its own domain. `time.js` gains `Time.signUp`, `Time.signIn`, `Time.signOut`, `Time.user` and `Time.mine`, and builds are told to use them for the paid tier's promise without ever putting sign-in in front of the free product.
|
||||
- Venture backends use Gemma 4 (`@cf/google/gemma-4-26b-a4b-it`) by default and read both the older `{response}` and the OpenAI-style answer from Workers AI (the first default had been retired).
|
||||
- Ventures can be backed by Cloudflare: with `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID`, each venture gets its own Worker, D1 database and Workers AI, and lives at `<slug>.<TIME_VENTURE_DOMAIN>` (or workers.dev) while its page stays on GitHub Pages. The page's `time.js` gives share links and a small model, and degrades to null so products still work without it. Existing ventures are backfilled a couple per tick; a new worker redeploys every venture; the doctor checks the token.
|
||||
- Venture loop launches products, not pages: a launch round where board seats write plans and one synthesizes `LAUNCH.md` with attribution; launch pages where the working product fills the first screen under a slim header with one call to action, then pricing and honest limits, in a look that follows the product; early-access signups through a GitHub issue form, thanked on their issues; deterministic and headless-Chrome checks with fix rounds and an andon; a 7-day decision against success and kill lines; a studio index at `/ventures/`.
|
||||
- Start a venture whenever you like: Actions → venture → Run workflow → "Start a new venture now" (`gh workflow run venture.yml -f start=true`); the schedule still starts one a day.
|
||||
- The board's taste: a delight anyone gets within a minute ranks above another developer utility, and when a product makes something people read (a deck, a card), its preloaded sample is the product's own pitch.
|
||||
- The studio index at `/ventures/` wears the Time site's blueprint theme (title block, grid paper, type), with each launched venture's laptop screenshot.
|
||||
- The headless-Chrome check fails a launch page where something sits on top of the product (a page fixed to the screen's height lets the next section slide over it).
|
||||
- The board sees the studio's own ventures: pitches that repeat one (by slug, name or an earlier slug) are set aside before the vote, since the week's trending list barely moves from day to day.
|
||||
- Hands-off upgrade support with spec planner, auto-merge, and Venice worker integration
|
||||
@@ -1,15 +1,24 @@
|
||||
# time
|
||||
|
||||
A GitHub-native agent factory. GitHub is the ledger, GitHub-hosted runners are the clock (Time), the worker is a Venice seat with hands or a Cursor cloud agent, and chat is where you plan.
|
||||
A Gitea-native agent factory, running on Teahub prod (teahub.xyz). The Gitea repo is the ledger, Gitea Actions are the clock (Time), the worker is a Claude or Venice seat with hands or a Cursor cloud agent, and chat is where you plan.
|
||||
|
||||
- **Cards**: an issue with a `DONE:` line, labelled `ready`.
|
||||
- **Heartbeat** (`heartbeat.yml`, hourly and whenever a card is labelled `ready`): moves cards forward. In `dry-run` (default) it only says what it would do; with `TIME_MODE=live` it dispatches each card to the worker (`TIME_WORKER`: a Venice seat such as `venice:kimi`, which edits the repo and opens the PR itself, or a Cursor cloud agent), which opens a PR that closes the card. An optional reviewer (`TIME_REVIEW`) checks the PR. At most `TIME_DAILY_CAP` launches a day.
|
||||
- **Mentions** (`mention.yml`): `@kimi`, `@grok 4.7`, `@qwen max high` in any issue, PR or comment gets that seat's reply (Venice).
|
||||
- **Heartbeat** (`heartbeat.yml`, hourly and whenever a card is labelled `ready`): moves cards forward. In `dry-run` (default) it only says what it would do; with `TIME_MODE=live` it dispatches each card to the worker (`TIME_WORKER`: a seat such as `claude:sonnet` or `venice:kimi`, which edits the repo and opens the PR itself, or a Cursor cloud agent), which opens a PR that closes the card. An optional reviewer (`TIME_REVIEW`) checks the PR. At most `TIME_DAILY_CAP` launches a day. **Spend governor:** every model call is booked to `time/ledger/<YYYY-MM>.jsonl` (tokens and dollars); once today's spend reaches `TIME_DAILY_USD` (default 5) or the month's reaches `TIME_MONTHLY_USD` (default 100), Time stops calling models, opens an `andon` issue, and a monthly breach also sets `TIME_MODE=dry-run`. No model decides whether to spend.
|
||||
- **Mentions** (`mention.yml`): `@sonnet`, `@opus`, `@kimi`, `@grok 4.7`, `@qwen max high` in any issue, PR or comment gets that seat's reply.
|
||||
- **Hands-off** (`TIME_AUTOMERGE=on`): label an issue `spec` and Time splits it into cards that run one at a time. A card's PR merges itself once the reviewer approves and checks are green, then the next card starts. Objections, red checks and protected paths (workflows, deploy, secrets) go to a human.
|
||||
- **Venture loop** (`venture.yml`, every 20 min once `TIME_VENTURE=on`): scouts the repos that gained the most stars this week, seats pitch one small static-page product each, the board votes (Borda), a seat builds it on `gh-pages`, and GitHub Pages ships it at `/ventures/<slug>/`. One card per venture (label `venture`), progress as comments.
|
||||
- **Doctor** (`doctor.yml`, manual): runs the tests and checks label, settings and keys.
|
||||
- **Venture loop** (`venture.yml`, every 20 min once `TIME_VENTURE=on`): one venture card a day, or one right away from Actions → venture → Run workflow → "Start a new venture now". It scouts the repos that gained the most stars this week; seats pitch; the board votes (Borda, quorum of two); every board seat writes a launch plan (positioning, first users, pricing, channels, 7-day success and kill lines) and one seat synthesizes them into `LAUNCH.md`; the worker builds the product as the page on `gh-pages` (the tool fills the first screen under a slim header with the one signup button; pricing, limits and the why sit below); deterministic checks and headless Chrome (phone and desktop) gate it, with two fix rounds before an andon; GitHub Pages ships it at `/ventures/<slug>/`. Signups go through an issue form (`.github/ISSUE_TEMPLATE/early-access.yml`), each one thanked on its issue; after 7 days the count of distinct people decides: build v2, kill, or watch a week more. The studio index lives at `/ventures/`. Launch posts are drafts for a human; nothing is posted outside the repo.
|
||||
- **Batches** (`TIME_VENTURE_BATCH=3`, or Run workflow → "batch"): one scout pass, a field of pitches, and the board's top k become venture cards in one milestone, built side by side and launched together. A held child gets one batch retry before it's counted dead; a retro compares the cohort once every child is measured.
|
||||
- **Seed** (`seed.yml`, manual): land a venture that was built somewhere else. Give it a slug, the URL of a .tgz whose top folder is that slug (or of its parts, in order) and the .tgz's sha256, plus a name and a one-liner for a new venture. `seed.py` checks the bundle, copies it to `ventures/<slug>/` on `gh-pages` in one commit, adds or refreshes its line in `ventures/ventures.json`, and asks Teahub pages to pull, so it is live at `<slug>.yolopreneur.lol` within the minute. Nothing in the bundle runs.
|
||||
- **Backends** (optional, `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID`): each venture also gets its own Cloudflare Worker, D1 database and Workers AI, serving its Pages folder at `<slug>.<TIME_VENTURE_DOMAIN>`. The page talks to it through `time.js` (share links that save a creation, a rate-limited small model, and accounts with passkeys: no passwords, no email); every call degrades gracefully, so products still work on Pages alone. Code: `.github/time/cloudflare.py` and `cloudflare/worker.js`.
|
||||
- **Doctor** (`doctor.yml`, manual): runs the tests and checks label, settings, keys and today's spend.
|
||||
- **Update** (`update.yml`, weekly or manual): opens a PR with the latest engine from teahubxyz/time, so every project stays on one Time instead of drifting copies.
|
||||
|
||||
## Setup
|
||||
|
||||
Open this repo in Claude Code or Cursor and say "set up Time". The agent follows `.claude/skills/time/SKILL.md`. You'll be asked to install Cursor's GitHub app on the repo and to paste your Cursor API key into `gh secret set CURSOR_API_KEY`.
|
||||
Started on GitHub as a proof of concept (teahubxyz/time, now read-only); ported to Gitea 2026-10-14.
|
||||
|
||||
Open this repo in Claude Code or Cursor and say "set up Time": the agent follows `.claude/skills/time/SKILL.md`. The recommended path needs one secret, `ANTHROPIC_API_KEY`: Claude seats plan, build and review (`TIME_WORKER=claude:sonnet`, `TIME_REVIEW=claude:haiku`; `haiku`, `sonnet`, `opus`, `fable` are the seats, priced in `.github/time/seats.json`). With `VENICE_API_KEY` instead, Venice seats do the same (`TIME_WORKER=venice:qwen coder`, `TIME_REVIEW=venice:grok`). A Cursor cloud agent can be the worker instead if you have a Cursor API key. The landing page walks through it: https://teahubxyz.github.io/time/
|
||||
|
||||
## Status
|
||||
|
||||
For changes and release history, see [CHANGELOG.md](CHANGELOG.md).
|
||||
@@ -0,0 +1,127 @@
|
||||
{"at":"2026-10-14T19:52:34Z","model":"claude-sonnet-5-5","in":1404,"out":71,"usd":0.003518}
|
||||
{"at":"2026-10-14T19:55:22Z","model":"claude-sonnet-5-5","in":1404,"out":71,"usd":0.003518}
|
||||
{"at":"2026-10-14T19:55:24Z","model":"claude-sonnet-5-5","in":1957,"out":196,"usd":0.005874}
|
||||
{"at":"2026-10-14T19:55:25Z","model":"claude-sonnet-5-5","in":5086,"out":61,"usd":0.010782}
|
||||
{"at":"2026-10-14T19:55:31Z","model":"claude-sonnet-5-5","in":6918,"out":790,"usd":0.021736}
|
||||
{"at":"2026-10-14T19:55:34Z","model":"claude-sonnet-5-5","in":7932,"out":230,"usd":0.018164}
|
||||
{"at":"2026-10-14T19:55:36Z","model":"claude-sonnet-5-5","in":8729,"out":171,"usd":0.019168}
|
||||
{"at":"2026-10-14T19:55:41Z","model":"claude-sonnet-5-5","in":8902,"out":497,"usd":0.022774}
|
||||
{"at":"2026-10-14T20:24:06Z","model":"claude-sonnet-5-5","in":1404,"out":71,"usd":0.003518}
|
||||
{"at":"2026-10-14T20:24:08Z","model":"claude-sonnet-5-5","in":1972,"out":173,"usd":0.005674}
|
||||
{"at":"2026-10-14T20:24:10Z","model":"claude-sonnet-5-5","in":4228,"out":137,"usd":0.009826}
|
||||
{"at":"2026-10-14T20:24:11Z","model":"claude-sonnet-5-5","in":5142,"out":121,"usd":0.011494}
|
||||
{"at":"2026-10-14T20:24:13Z","model":"claude-sonnet-5-5","in":5270,"out":81,"usd":0.01135}
|
||||
{"at":"2026-10-14T20:24:14Z","model":"claude-sonnet-5-5","in":5477,"out":135,"usd":0.012304}
|
||||
{"at":"2026-10-14T20:24:16Z","model":"claude-sonnet-5-5","in":6192,"out":105,"usd":0.013434}
|
||||
{"at":"2026-10-14T20:24:18Z","model":"claude-sonnet-5-5","in":6504,"out":245,"usd":0.015458}
|
||||
{"at":"2026-10-14T20:24:22Z","model":"claude-sonnet-5-5","in":7887,"out":475,"usd":0.020524}
|
||||
{"at":"2026-10-14T20:24:30Z","model":"claude-sonnet-5-5","in":8409,"out":844,"usd":0.025258}
|
||||
{"at":"2026-10-14T20:26:11Z","model":"claude-haiku-5-5","in":1660,"out":1406,"usd":0.000869}
|
||||
{"at":"2026-10-14T21:03:34Z","model":"claude-sonnet-5-5","in":1541,"out":79,"usd":0.003872}
|
||||
{"at":"2026-10-14T21:03:42Z","model":"claude-sonnet-5-5","in":29380,"out":970,"usd":0.06846}
|
||||
{"at":"2026-10-14T21:03:48Z","model":"claude-sonnet-5-5","in":30236,"out":267,"usd":0.063142}
|
||||
{"at":"2026-10-14T21:03:53Z","model":"claude-sonnet-5-5","in":30731,"out":639,"usd":0.067852}
|
||||
{"at":"2026-10-14T21:08:04Z","model":"claude-haiku-5-5","in":1532,"out":1201,"usd":0.000754}
|
||||
{"at":"2026-10-14T21:18:09Z","model":"claude-sonnet-5-5","in":1449,"out":26,"usd":0.003158}
|
||||
{"at":"2026-10-14T21:18:11Z","model":"claude-sonnet-5-5","in":1928,"out":100,"usd":0.004856}
|
||||
{"at":"2026-10-14T21:18:13Z","model":"claude-sonnet-5-5","in":2470,"out":107,"usd":0.00601}
|
||||
{"at":"2026-10-14T21:18:15Z","model":"claude-sonnet-5-5","in":5197,"out":151,"usd":0.011904}
|
||||
{"at":"2026-10-14T21:18:17Z","model":"claude-sonnet-5-5","in":8086,"out":211,"usd":0.018282}
|
||||
{"at":"2026-10-14T21:18:19Z","model":"claude-sonnet-5-5","in":10670,"out":136,"usd":0.0227}
|
||||
{"at":"2026-10-14T21:18:24Z","model":"claude-sonnet-5-5","in":11881,"out":438,"usd":0.028142}
|
||||
{"at":"2026-10-14T21:18:35Z","model":"claude-sonnet-5-5","in":13013,"out":1796,"usd":0.043986}
|
||||
{"at":"2026-10-14T21:18:44Z","model":"claude-sonnet-5-5","in":14752,"out":1511,"usd":0.044614}
|
||||
{"at":"2026-10-14T21:18:51Z","model":"claude-sonnet-5-5","in":16199,"out":460,"usd":0.036998}
|
||||
{"at":"2026-10-14T21:22:16Z","model":"claude-opus-5-5","in":8564,"out":713,"usd":0.048516}
|
||||
{"at":"2026-10-14T21:23:11Z","model":"claude-haiku-5-5","in":2858,"out":1705,"usd":0.001138}
|
||||
{"at":"2026-10-14T21:39:57Z","model":"claude-sonnet-5-5","in":1527,"out":26,"usd":0.003314}
|
||||
{"at":"2026-10-14T21:39:59Z","model":"claude-sonnet-5-5","in":2006,"out":94,"usd":0.004952}
|
||||
{"at":"2026-10-14T21:40:00Z","model":"claude-sonnet-5-5","in":4528,"out":54,"usd":0.009596}
|
||||
{"at":"2026-10-14T21:40:03Z","model":"claude-sonnet-5-5","in":28923,"out":182,"usd":0.059666}
|
||||
{"at":"2026-10-14T21:40:06Z","model":"claude-sonnet-5-5","in":30029,"out":254,"usd":0.062598}
|
||||
{"at":"2026-10-14T21:40:09Z","model":"claude-sonnet-5-5","in":33924,"out":187,"usd":0.069718}
|
||||
{"at":"2026-10-14T21:40:12Z","model":"claude-sonnet-5-5","in":39941,"out":108,"usd":0.080962}
|
||||
{"at":"2026-10-14T21:40:15Z","model":"claude-sonnet-5-5","in":42992,"out":246,"usd":0.088444}
|
||||
{"at":"2026-10-14T21:40:17Z","model":"claude-sonnet-5-5","in":46955,"out":146,"usd":0.09537}
|
||||
{"at":"2026-10-14T21:41:13Z","model":"claude-sonnet-5-5","in":49497,"out":6680,"usd":0.165794}
|
||||
{"at":"2026-10-14T21:42:37Z","model":"claude-sonnet-5-5","in":50026,"out":10161,"usd":0.201662}
|
||||
{"at":"2026-10-14T21:44:19Z","model":"claude-sonnet-5-5","in":51291,"out":13054,"usd":0.233122}
|
||||
{"at":"2026-10-14T21:45:26Z","model":"claude-sonnet-5-5","in":54098,"out":9726,"usd":0.205456}
|
||||
{"at":"2026-10-14T21:45:34Z","model":"claude-sonnet-5-5","in":59375,"out":708,"usd":0.12583}
|
||||
{"at":"2026-10-14T21:46:19Z","model":"claude-sonnet-5-5","in":59860,"out":6021,"usd":0.17993}
|
||||
{"at":"2026-10-14T21:46:23Z","model":"claude-sonnet-5-5","in":63582,"out":383,"usd":0.130994}
|
||||
{"at":"2026-10-14T21:46:31Z","model":"claude-sonnet-5-5","in":64284,"out":690,"usd":0.135468}
|
||||
{"at":"2026-10-14T21:46:37Z","model":"claude-sonnet-5-5","in":64859,"out":488,"usd":0.134598}
|
||||
{"at":"2026-10-14T21:46:45Z","model":"claude-sonnet-5-5","in":65253,"out":650,"usd":0.137006}
|
||||
{"at":"2026-10-14T21:46:52Z","model":"claude-sonnet-5-5","in":65768,"out":637,"usd":0.137906}
|
||||
{"at":"2026-10-14T22:08:11Z","model":"claude-haiku-5-5","in":11278,"out":2444,"usd":0.00235}
|
||||
{"at":"2026-10-15T13:13:31Z","model":"claude-opus-5-5","in":23639,"out":614,"usd":0.106836}
|
||||
{"at":"2026-10-15T21:07:27Z","model":"claude-sonnet-5-5","in":1824,"out":26,"usd":0.003908}
|
||||
{"at":"2026-10-15T21:07:30Z","model":"claude-sonnet-5-5","in":2303,"out":172,"usd":0.006326}
|
||||
{"at":"2026-10-15T21:07:32Z","model":"claude-sonnet-5-5","in":2899,"out":154,"usd":0.007338}
|
||||
{"at":"2026-10-15T21:07:35Z","model":"claude-sonnet-5-5","in":5409,"out":257,"usd":0.013388}
|
||||
{"at":"2026-10-15T21:07:38Z","model":"claude-sonnet-5-5","in":7805,"out":223,"usd":0.01784}
|
||||
{"at":"2026-10-15T21:07:41Z","model":"claude-sonnet-5-5","in":10371,"out":169,"usd":0.022432}
|
||||
{"at":"2026-10-15T21:07:46Z","model":"claude-sonnet-5-5","in":11385,"out":395,"usd":0.02672}
|
||||
{"at":"2026-10-15T21:07:50Z","model":"claude-sonnet-5-5","in":13978,"out":300,"usd":0.030956}
|
||||
{"at":"2026-10-15T21:07:57Z","model":"claude-sonnet-5-5","in":16724,"out":661,"usd":0.040058}
|
||||
{"at":"2026-10-15T21:08:04Z","model":"claude-sonnet-5-5","in":17799,"out":493,"usd":0.040528}
|
||||
{"at":"2026-10-15T21:08:09Z","model":"claude-sonnet-5-5","in":18178,"out":229,"usd":0.038646}
|
||||
{"at":"2026-10-15T21:08:12Z","model":"claude-sonnet-5-5","in":19392,"out":121,"usd":0.039994}
|
||||
{"at":"2026-10-15T21:08:16Z","model":"claude-sonnet-5-5","in":19681,"out":279,"usd":0.042152}
|
||||
{"at":"2026-10-15T21:08:22Z","model":"claude-sonnet-5-5","in":20505,"out":168,"usd":0.04269}
|
||||
{"at":"2026-10-15T21:08:29Z","model":"claude-sonnet-5-5","in":22991,"out":158,"usd":0.047562}
|
||||
{"at":"2026-10-15T21:08:34Z","model":"claude-sonnet-5-5","in":25613,"out":290,"usd":0.054126}
|
||||
{"at":"2026-10-15T21:08:41Z","model":"claude-sonnet-5-5","in":29778,"out":279,"usd":0.062346}
|
||||
{"at":"2026-10-15T21:08:48Z","model":"claude-sonnet-5-5","in":31149,"out":220,"usd":0.064498}
|
||||
{"at":"2026-10-15T21:08:55Z","model":"claude-sonnet-5-5","in":31813,"out":316,"usd":0.066786}
|
||||
{"at":"2026-10-15T21:09:04Z","model":"claude-sonnet-5-5","in":33214,"out":175,"usd":0.068178}
|
||||
{"at":"2026-10-15T21:09:11Z","model":"claude-sonnet-5-5","in":35083,"out":239,"usd":0.072556}
|
||||
{"at":"2026-10-15T21:09:25Z","model":"claude-sonnet-5-5","in":36773,"out":1171,"usd":0.085256}
|
||||
{"at":"2026-10-15T21:09:33Z","model":"claude-sonnet-5-5","in":38114,"out":680,"usd":0.083028}
|
||||
{"at":"2026-10-15T21:09:43Z","model":"claude-sonnet-5-5","in":40762,"out":520,"usd":0.086724}
|
||||
{"at":"2026-10-15T21:09:47Z","model":"claude-sonnet-5-5","in":43289,"out":163,"usd":0.088208}
|
||||
{"at":"2026-10-15T21:09:54Z","model":"claude-sonnet-5-5","in":45917,"out":617,"usd":0.098004}
|
||||
{"at":"2026-10-15T21:10:10Z","model":"claude-sonnet-5-5","in":47228,"out":1790,"usd":0.112356}
|
||||
{"at":"2026-10-15T21:10:14Z","model":"claude-sonnet-5-5","in":48839,"out":100,"usd":0.098678}
|
||||
{"at":"2026-10-15T21:10:18Z","model":"claude-sonnet-5-5","in":51169,"out":364,"usd":0.105978}
|
||||
{"at":"2026-10-15T21:10:22Z","model":"claude-sonnet-5-5","in":52631,"out":115,"usd":0.106412}
|
||||
{"at":"2026-10-15T21:11:02Z","model":"claude-sonnet-5-5","in":53793,"out":4582,"usd":0.153406}
|
||||
{"at":"2026-10-15T21:11:14Z","model":"claude-sonnet-5-5","in":56077,"out":1296,"usd":0.125114}
|
||||
{"at":"2026-10-15T21:12:05Z","model":"claude-sonnet-5-5","in":56783,"out":3167,"usd":0.145236}
|
||||
{"at":"2026-10-15T21:12:22Z","model":"claude-sonnet-5-5","in":59396,"out":1933,"usd":0.138122}
|
||||
{"at":"2026-10-15T21:13:04Z","model":"claude-sonnet-5-5","in":59720,"out":3191,"usd":0.15135}
|
||||
{"at":"2026-10-15T21:13:29Z","model":"claude-sonnet-5-5","in":60806,"out":2548,"usd":0.147092}
|
||||
{"at":"2026-10-15T21:14:48Z","model":"claude-sonnet-5-5","in":61169,"out":3172,"usd":0.154058}
|
||||
{"at":"2026-10-15T21:15:32Z","model":"claude-sonnet-5-5","in":61526,"out":294,"usd":0.125992}
|
||||
{"at":"2026-10-15T21:16:46Z","model":"claude-sonnet-5-5","in":63016,"out":3974,"usd":0.165772}
|
||||
{"at":"2026-10-15T21:17:56Z","model":"claude-sonnet-5-5","in":64261,"out":8283,"usd":0.211352}
|
||||
{"at":"2026-10-15T21:18:27Z","model":"claude-sonnet-5-5","in":1910,"out":26,"usd":0.00408}
|
||||
{"at":"2026-10-15T21:18:32Z","model":"claude-sonnet-5-5","in":2389,"out":127,"usd":0.006048}
|
||||
{"at":"2026-10-15T21:18:37Z","model":"claude-sonnet-5-5","in":2820,"out":178,"usd":0.00742}
|
||||
{"at":"2026-10-15T21:18:43Z","model":"claude-sonnet-5-5","in":3412,"out":200,"usd":0.008824}
|
||||
{"at":"2026-10-15T21:18:49Z","model":"claude-sonnet-5-5","in":8239,"out":200,"usd":0.018478}
|
||||
{"at":"2026-10-15T21:19:15Z","model":"claude-sonnet-5-5","in":9837,"out":311,"usd":0.022784}
|
||||
{"at":"2026-10-15T21:19:21Z","model":"claude-sonnet-5-5","in":10098,"out":318,"usd":0.023376}
|
||||
{"at":"2026-10-15T21:19:48Z","model":"claude-sonnet-5-5","in":10435,"out":488,"usd":0.02575}
|
||||
{"at":"2026-10-15T21:20:27Z","model":"claude-sonnet-5-5","in":13110,"out":1545,"usd":0.04167}
|
||||
{"at":"2026-10-15T21:21:24Z","model":"claude-sonnet-5-5","in":18362,"out":1083,"usd":0.047554}
|
||||
{"at":"2026-10-15T21:22:03Z","model":"claude-sonnet-5-5","in":20427,"out":1811,"usd":0.058964}
|
||||
{"at":"2026-10-15T21:22:09Z","model":"claude-sonnet-5-5","in":22877,"out":174,"usd":0.047494}
|
||||
{"at":"2026-10-15T21:22:13Z","model":"claude-sonnet-5-5","in":23372,"out":117,"usd":0.047914}
|
||||
{"at":"2026-10-15T21:22:17Z","model":"claude-sonnet-5-5","in":25793,"out":120,"usd":0.052786}
|
||||
{"at":"2026-10-15T21:22:20Z","model":"claude-sonnet-5-5","in":27604,"out":135,"usd":0.056558}
|
||||
{"at":"2026-10-15T21:22:26Z","model":"claude-sonnet-5-5","in":30017,"out":195,"usd":0.061984}
|
||||
{"at":"2026-10-15T21:22:31Z","model":"claude-sonnet-5-5","in":33384,"out":172,"usd":0.068488}
|
||||
{"at":"2026-10-15T21:22:36Z","model":"claude-sonnet-5-5","in":35727,"out":234,"usd":0.073794}
|
||||
{"at":"2026-10-15T21:22:42Z","model":"claude-sonnet-5-5","in":38098,"out":235,"usd":0.078546}
|
||||
{"at":"2026-10-15T21:22:48Z","model":"claude-sonnet-5-5","in":39963,"out":223,"usd":0.082156}
|
||||
{"at":"2026-10-15T21:22:55Z","model":"claude-sonnet-5-5","in":41469,"out":389,"usd":0.086828}
|
||||
{"at":"2026-10-15T21:23:02Z","model":"claude-sonnet-5-5","in":43852,"out":563,"usd":0.093334}
|
||||
{"at":"2026-10-15T21:23:11Z","model":"claude-sonnet-5-5","in":45712,"out":299,"usd":0.094414}
|
||||
{"at":"2026-10-15T21:23:18Z","model":"claude-sonnet-5-5","in":48051,"out":359,"usd":0.099692}
|
||||
{"at":"2026-10-15T21:23:25Z","model":"claude-sonnet-5-5","in":49292,"out":591,"usd":0.104494}
|
||||
{"at":"2026-10-15T21:23:32Z","model":"claude-sonnet-5-5","in":49954,"out":535,"usd":0.105258}
|
||||
{"at":"2026-10-15T21:23:47Z","model":"claude-sonnet-5-5","in":50931,"out":1659,"usd":0.118452}
|
||||
{"at":"2026-10-15T21:24:16Z","model":"claude-sonnet-5-5","in":53002,"out":241,"usd":0.108414}
|
||||
{"at":"2026-10-15T21:25:18Z","model":"claude-sonnet-5-5","in":55414,"out":858,"usd":0.119408}
|
||||
@@ -0,0 +1,30 @@
|
||||
# Spend ledger
|
||||
|
||||
Time keeps a record of what it spends on models. This folder holds it.
|
||||
|
||||
- **One file per month:** `time/ledger/<YYYY-MM>.jsonl`, for example `2026-10.jsonl`.
|
||||
- **One JSON line per model call.** Fields:
|
||||
- `at`: when the call was made, ISO 8601 in UTC (`2026-10-14T09:30:00Z`).
|
||||
- `model`: the model that was called.
|
||||
- `in`: input (prompt) tokens.
|
||||
- `out`: output (completion) tokens.
|
||||
- `usd`: the cost in US dollars, or `null` if the seat has no price (unpriced seats).
|
||||
|
||||
Example line:
|
||||
|
||||
```json
|
||||
{"at":"2026-10-14T09:30:00Z","model":"some-model","in":1200,"out":350,"usd":0.0123}
|
||||
```
|
||||
|
||||
## Who writes and reads it
|
||||
|
||||
- **Written by Time itself.** At the end of each run, Time appends that run's calls to the current month's file in one `ledger:` commit.
|
||||
- **Read before every model call.** Whichever workflow is about to call a model (heartbeat, mentions, venture loop, Astra) adds up `usd` for today and for the month and refuses the call if either total has reached its cap (`TIME_DAILY_USD`, `TIME_MONTHLY_USD`). A `null` counts as zero.
|
||||
|
||||
## When a cap trips
|
||||
|
||||
Time stops calling models. The next time the heartbeat has a card to run, it opens one issue labelled `andon` (never a second while that one is open). A monthly cap also switches `TIME_MODE` to `dry-run`, or says in the issue that it couldn't, until a human raises the cap. A daily cap clears itself at midnight UTC. Close the issue once it is handled.
|
||||
|
||||
## Don't edit it by hand
|
||||
|
||||
Humans should never edit these files. The caps rely on them being exact, and Time only ever appends. A hand edit can hide spend or break the next append.
|
||||
Reference in new issue
Block a user