Plans, facts and skills, no tokens.
When the steps are known, an agent needs no model. A skill file replays them and checks each one, and a model is asked only from the first step that fails.
Replay, check, then ask
A skill is a file of checked steps.
The skills harness turns a run that succeeded into a skill file: a phrase, its parameters, and each computer_act step with what the screen must show after it.
Case study 1 · a simulated desktop
The same task: 0 calls against 17.
The sim_save task on a simulated desktop. The scripted agent sends the actions a correct agent sends; the baseline sends each screenshot to a vision model and takes its one action.
- passed by the scripted agent, 0 model calls, $0, 2.4 s in all
- 3 of 3
- passed by the vision-model baseline, 17 calls, $0.0013, 124 s
- 1 of 1
Simulated
test-results/computer-eval/scripted-serverless-2026-10-28T06-27-57-260Z.json and baseline-serverless-2026-10-28T06-30-05-354Z.json (this computer)输出 case-studies/computer-evals/stats.json工具 scripts/workbench/computer-eval.mjs, read by scripts/sites/case-studies.mjs computer-evals运行于 This computer (dell-gb10-1), a simulated desktop (ComputerWorld)时间 Case study 2 · skills on ten tasks
Skills pass as often, in a quarter of the time.
Ten desktop tasks, two runs each. The skills agent replays what it learned and asks a model only where a step fails.
Not zero yet. On these ten tasks the skills agent still called a model where a step failed. The runs with no model call at all are the scripted agent's, above.
- passed by the skills agent, with 114 model calls and 443 s
- 11 of 20
- passed by the vision-model baseline, with 282 calls and 1,729 s
- 10 of 20
- for the skills agent (deepseek-flash) and $0.019 for the baseline (qwen3.7-flash): fewer calls, but not cheaper here
- $0.057
test-results/computer-eval/skills-2026-10-25T01-49-30-309Z.json and baseline-2026-10-25T02-39-16-279Z.json (this computer)输出 case-studies/computer-evals/stats.json工具 scripts/workbench/computer-eval.mjs --harness skills运行于 This computer (dell-gb10-1), a private Xvfb desktop时间 Teach it once, replay it free.
Run a task once with a model, and the skills harness keeps the steps.