Work goes to the agent that does it best.
A finished task is rated for its outcome and its review. The ratings fill a pool of task regions, and each new task goes to the member with the best expected quality per dollar.
From a rating to a route
A rating is one line in the tasks stream.
The rated event names the task, the configuration that did it (CLI, route, model, effort), the rater, and a score with its precision, confidence and importance.
Case study · fitted on our own ratings
On held-out tasks, 71% chose a best agent.
The backlog thread's ratings, oldest 70% learned, the rest held out. A held-out task that two or more configurations did is routed as if new, and the choice is checked against the one that did best per dollar. No model ran in the fit.
What the numbers say. The routing fit beats the shipped constants. The calibration fit does not beat the plain mean yet, so the regions do not carry a signal on our tasks. The fitter recommends no change to the calibration, and says so.
- ratings: 2,779 by the task gates, 618 by model reviewers
- 3,397
- of 24 held-out tasks routed to a best configuration, at the best constants (prior weight 8, no exploration)
- 71%
- at the shipped constants (prior weight 3, exploration 0.3)
- 58%
- mean error predicting a held-out score, against 0.152 for the plain mean
- 0.164
runs/20261023t132208z-0aea (the backlog thread, on this computer)Output case-studies/routing-by-rating/stats.jsonTool scripts/agents/fit-routing.mjs, through scripts/sites/case-studies.mjs routing-by-ratingRan on This computer (dell-gb10-1)When The owner's priors against our ratings
The ratings disagree with the prior.
The shipped routing says Codex is better at abstract work (0.75 against 0.55). In our rated tasks, Claude Code scored higher in every region. A configuration here is the CLI with its route, and most of Claude Code's turns ran DeepSeek through it, so this compares harnesses on our tasks, not models.
Codex has few ratings here, so its numbers are weak. The fit recommends prior weight 8 and no exploration for routing; the shipped constants have not been changed to them yet.
- Claude Code's mean quality on abstract work (878 weighted ratings)
- 0.76
- Codex's on abstract work (19)
- 0.28
- on clean-up and review (192 and 9)
- 0.67 and 0.27
runs/20261023t132208z-0aea (the backlog thread, on this computer)Output case-studies/routing-by-rating/stats.jsonTool scripts/agents/fit-routing.mjs (measuredPriors)Ran on This computer (dell-gb10-1)When Let the ratings pick the agent.
Rate a finished task in the thread, or let the task gates rate it, and the next task routes by it.