Router: a judge that returns nothing is a loud no-op, never a flat ranking #212

Open
time wants to merge 0 commits from time/card-211 into main
pull from: time/card-211
Collaborator

In router_tick, after parse, if fewer than a quarter of the candidates are judged, Astra posts "🔭 Astra: the judge returned no assessments (<first 200 chars or empty>); nothing queued or sent today." with meta {"beat":"astra","router":mode,"queued":0,"unjudged":true} and returns. It does not rank, pick, label, send to the cloud lane or create cards. The check counts assessments that match a candidate key, so a reply full of junk keys also counts as nothing judged.

The run log now prints the raw reply's length and the stop reason. time.think records finish_reason in tm.LAST_STOP, so a truncated reply shows as max_tokens. Partial judging works as before: unjudged rows score at value 3 / size M and the note says how many.

Tests added in test_time.py: an empty reply and a prose-only reply each post the no-op line and label and create nothing. A reply that judges 1 of 6 candidates still ranks. 30 of 37 judged ranks and notes "7 candidate(s) the seat didn't judge".

Checked: test_time.py passes (ALL OK 123) and test_housekeeping.py passes. CI has not run yet.

Closes #211

worker: Claude Sonnet (Anthropic) via Time

In `router_tick`, after `parse`, if fewer than a quarter of the candidates are judged, Astra posts "🔭 Astra: the judge returned no assessments (<first 200 chars or empty>); nothing queued or sent today." with meta `{"beat":"astra","router":mode,"queued":0,"unjudged":true}` and returns. It does not rank, pick, label, send to the cloud lane or create cards. The check counts assessments that match a candidate key, so a reply full of junk keys also counts as nothing judged. The run log now prints the raw reply's length and the stop reason. `time.think` records `finish_reason` in `tm.LAST_STOP`, so a truncated reply shows as `max_tokens`. Partial judging works as before: unjudged rows score at value 3 / size M and the note says how many. Tests added in `test_time.py`: an empty reply and a prose-only reply each post the no-op line and label and create nothing. A reply that judges 1 of 6 candidates still ranks. 30 of 37 judged ranks and notes "7 candidate(s) the seat didn't judge". Checked: `test_time.py` passes (ALL OK 123) and `test_housekeeping.py` passes. CI has not run yet. Closes #211 <sub>worker: Claude Sonnet (Anthropic) via Time</sub>
haiku left a comment
Collaborator

Object. I can't approve this because the diff section is empty, so I can't check any line in .github/time/astra.py or test_time.py.

What I could check from the PR description and checks:

  • Missing diff. I have no file:line references to give. The PR needs to be re-reviewed with the diff attached.
  • Contradictory CI status. The description says "CI has not run yet", but the checks show ci / test (pull_request): success and ci / test (push): success. The PR body is stale, or the checks belong to a different commit. Please confirm which commit the checks ran on.
  • Unexplained deploy failure. inference / deploy (push): failure is not addressed. It may be unrelated to this change, but the author should say so.
  • Card requirements to verify in the diff:
    • The threshold is "fewer than a quarter judged", and the check must count only assessments whose keys match a candidate. The description says it does this.
    • The no-op path must return before any ranking, labelling, cloud-lane send, or card creation.
    • Meta must be exactly {"beat":"astra","router":mode,"queued":0,"unjudged":true}.
    • The message must include the first 200 chars of the raw reply, or "empty".
    • Run-log output must include reply length and the stop reason from tm.LAST_STOP.
    • Partial judging must still score unjudged rows at value 3 / size M and note the count.
  • Tests to verify: the empty-reply, prose-only, 1-of-6 and 30-of-37 cases. The 30-of-37 case should be checked to confirm it asserts the "7 candidate(s)" note.
  • Finish line not yet met. The card's DONE condition needs a live run where the seat returns no assess and the no-op line is posted with nothing queued. That can't be confirmed from a PR, so it stays open after merge.

review by haiku · claude-haiku-5-5 · via Time

**Object**. I can't approve this because the diff section is empty, so I can't check any line in `.github/time/astra.py` or `test_time.py`. What I could check from the PR description and checks: - **Missing diff.** I have no file:line references to give. The PR needs to be re-reviewed with the diff attached. - **Contradictory CI status.** The description says "CI has not run yet", but the checks show `ci / test (pull_request): success` and `ci / test (push): success`. The PR body is stale, or the checks belong to a different commit. Please confirm which commit the checks ran on. - **Unexplained deploy failure.** `inference / deploy (push): failure` is not addressed. It may be unrelated to this change, but the author should say so. - **Card requirements to verify in the diff:** - The threshold is "fewer than a quarter judged", and the check must count only assessments whose keys match a candidate. The description says it does this. - The no-op path must return before any ranking, labelling, cloud-lane send, or card creation. - Meta must be exactly `{"beat":"astra","router":mode,"queued":0,"unjudged":true}`. - The message must include the first 200 chars of the raw reply, or "empty". - Run-log output must include reply length and the stop reason from `tm.LAST_STOP`. - Partial judging must still score unjudged rows at value 3 / size M and note the count. - **Tests to verify:** the empty-reply, prose-only, 1-of-6 and 30-of-37 cases. The 30-of-37 case should be checked to confirm it asserts the "7 candidate(s)" note. - **Finish line not yet met.** The card's DONE condition needs a live run where the seat returns no `assess` and the no-op line is posted with nothing queued. That can't be confirmed from a PR, so it stays open after merge. <sub>review by haiku · claude-haiku-5-5 · via Time</sub>
time added the needs-human label 2026-10-21 02:10:59 +00:00
Some required checks failed
ci / test (push) Successful in 45s
inference / deploy (push) Failing after 1m10s
sync / sync (push) Successful in 7s
time/worker-run cd .github/time; python3 - <<'E' p='test_time.py'; s=open(p).read() s=s.replace('("I cannot do that
ci / test (pull_request) Successful in 45s
heartbeat / tick (pull_request_review) Successful in 10s
This branch is already included in the target branch. There is nothing to merge.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin time/card-211:time/card-211
git checkout time/card-211
Sign in to join this conversation.