Field report · 2 min read
A stopped mission kept spending tokens
Public benchmark trials exposed a running process that survived mission withdrawal and consumed more than a million tokens on unusable work.
A withdrawn Grex mission continued answering benchmark questions for more than two hours. Its own run budget finally stopped it after 1,492,130 tokens across 440 calls.
The work could no longer count toward a mission result. This was a failure of stopping behavior discovered during tests of agent capability.
Grex runs AI assignments with budgets, evidence requirements, and defined completion rules. Today, it used that same mission process to run samples from public agent benchmarks.
What the benchmark samples measured
The worker computers loaded pinned tasks, solved them, and sent answers to a separate scoring run. They then published reports through the ordinary mission workflow.
| Benchmark sample | Recorded result | Limitation |
|---|---|---|
| GAIA, general assistant tasks | 21 passes among 27 scored | Thirty selected; two timed out and one was refused by the model |
| tau-bench, simulated retail tasks | 16 passes out of twenty | A limited retail sample |
| MCP-Atlas, tool-use tasks | Zero passes out of three | Required tool servers could not start |
A frontier model solved these tasks. The results were a reference for that model operating in Grex, not evidence of how much Grex improved it.
Why withdrawal failed to stop the work
A scheduled work claim survived the mission’s withdrawal. A later process picked it up without the original mission terms and fell back to a broader task set.
The running process also survived stop and permission-expiry events. A renewal bug prevented its execution permission from expiring when renewals were refused.
The missing terms affected answer quality too. One previously correct species answer became the less specific “Penguin.”
The repair had to reach the running process
The central service was changed to reject work and publication for closed missions. The worker was changed to terminate running processes on stop, expiry, timeout, and shutdown, with a forced kill after a grace period.
The work also prompted a clearer split between strategic judgment and routine execution. Missions needed a supervisor to reason about progress, and records needed to identify which model performed each role.
The next planned comparison would put a budget model in the solving role. Before interpreting any capability score, however, Grex had to make “stop” mean that execution actually stopped.