labs/aicharts: decision confidences + per-request floor for calibration #41

Merged

teahubxyz merged 1 commits from labs/aicharts-calibrate into main

2026-10-13 15:31:06 +00:00

Author	SHA1	Message	Date
Claude	b11edd20e8	labs/aicharts: show decision confidences, per-request floor, bench user agent Live: Clef answered both probes right in ~0.5 s but at 0.21-0.25 confidence, so the 0.5 floor sent them to the LLM. Expose every confidence and the weakest question; a request may set its own floor (only changes cost); bench.py takes clef@0 for calibration and sends a user agent Cloudflare accepts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VDkN7ukK1kXkqov8FaZxva	2026-10-13 15:29:59 +00:00

Author

SHA1

Message

Date

Claude

b11edd20e8

labs/aicharts: show decision confidences, per-request floor, bench user agent

Live: Clef answered both probes right in ~0.5 s but at 0.21-0.25 confidence, so
the 0.5 floor sent them to the LLM. Expose every confidence and the weakest
question; a request may set its own floor (only changes cost); bench.py takes
clef@0 for calibration and sends a user agent Cloudflare accepts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDkN7ukK1kXkqov8FaZxva

2026-10-13 15:29:59 +00:00