Historical Grex 1 · October 2026
Did the extra execution layers help?
Five historical pairs scored equally overall, leading to a simpler path for one bounded task.
Completed · historical architecture
Across five pairs, direct execution and the earlier Grex chain each passed 17 of 41 judged rubric items. Individual pairs differed, and the judging had known limitations. The decision was to simplify single-task execution while keeping mission-level controls outside the experiment’s scope.
The experiment, explained
A narrated explanation of recorded results, with synthetic narration and visible English captions. Diagrams explain the method; they are not recordings of execution.
Read the video transcript
Did the extra layers help?
For a task that needs one solver invocation, do extra execution layers improve the result? We compared a direct path with the longer chain in the earlier Grex architecture.
A narrower hypothesis
The recorded question was whether direct execution with frozen inputs and independent verification could do as well. This is a retrospective hypothesis summary, not a registered numerical prediction.
Five pairs, three task types
Five pairs covered three task types and two models. Each pair shared its task, model and prompt. Local rubric judging used three judgments per item and the median result.
An aggregate tie
Both paths passed seventeen of forty-one judged items. These are rubric items, not seventeen completed tasks. In the holdout, both produced files but timed out and scored poorly.
The limits stayed visible
Repeated development tasks, environment differences and provenance-label failures limited the comparison. The judging was local, with known defects. This was not an official JobBench result or a current-platform score.
Simplify the bounded invocation
The decision was to simplify execution of one bounded task. Scheduling, persistence, authorization, recovery and cooperating agents were outside this comparison. The result did not show those responsibilities were unnecessary.
What were we trying to find out?
For a single bounded task, does the earlier Grex execution chain improve the output beyond one direct solver invocation?
Hypothesis
One direct invocation with frozen inputs and independent verification might perform as well as the longer execution chain.
The adopted architecture decision records this as the experiment question. This directional wording is a retrospective summary, not a separately verified preregistered numerical prediction.
How the experiment worked
Run the same frozen task, model and prompt once directly and once through the earlier Grex invocation chain. Compare the retained deliverables using local rubric judging. A later pair used a previously uninspected task.
Direct
- One solver invocation
- Retain deliverables and execution facts
- Independent rubric judgment
Earlier Grex chain
- Dispatcher and task-wrapper chain
- Same solver and retained artifacts
- Same rubric judgment
17 of 41 judged items in each path
Five pairs; environment and judging defects; not official JobBench or a current-platform score.
Download the method illustration
Sample: Five pairs across three task types and two solver models. Repeated biostatistics pairs are not independent task samples. Scores count rubric items, not fully completed tasks.
Freeze each task
Retain task inputs, prompt, model and expected deliverables for each pair.
Use two execution paths
Invoke the solver directly or through the earlier Grex process chain.
Retain every outcome
Keep artifacts, timeouts, integrity checks and provenance failures.
Judge the deliverables
Use a local Gemini-backed rubric judge with three judgments per item and the median outcome.
Controls and differences
- Both arms in a pair used the same frozen task, model and prompt.
- One Sonnet pair reversed execution order; that pair also changed a Grex source pin, preventing clean order attribution.
- The bookkeeping holdout was selected by a fixed rule before its task card, rubric and references were inspected.
- Execution and artifact integrity were recorded separately from rubric judgments.
What happened
| Measure | Direct invocation | Earlier Grex chain |
|---|---|---|
| Biostatistics · Sonnet | 4 / 8 items | 4 / 8 items |
| Biostatistics · MiniMax | 4 / 8 items | 2 / 8 items |
| Biostatistics · Sonnet, reversed order | 4 / 8 items | 5 / 8 items |
| Administrative task · MiniMax | 3 / 7 items | 4 / 7 items |
| Bookkeeping holdout · MiniMax | 2 / 10 items | 2 / 10 items |
| Pooled rubric items | 17 / 41 | 17 / 41 |
The aggregate was tied
Both paths passed 17 of 41 judged items. The largest pair-level gap was two items. This found no measured overall output advantage for the extra chain on these tasks.
The holdout was difficult for both
Both bookkeeping attempts produced the three expected files but timed out and returned no final text. Each passed two of ten judged items. Producing files was not equivalent to completing the task correctly.
The experiment exposed comparison defects
Effective environments were not fully equal, and some chain attempts were marked failed because of provenance-label mismatch despite retained artifacts passing integrity checks. Those failures and mismatches remain part of the record.
The decision had a narrow scope
The resulting direction used one bounded solver invocation for one task. Scheduling, persistence, authorization, recovery and multi-agent coordination were not tested by this comparison and were not shown unnecessary.
What this does not establish
- This concerns the earlier Grex architecture, not a present-day Grex 2 benchmark score.
- Five pairs across three task types are too few for a general equivalence or superiority claim.
- Local rubric judging had known validity defects and was not official JobBench evaluation.
- Two task types had been inspected during development. Repeated tasks, model variation and environment differences limit causal inference.
- No general speed advantage or complete all-in cost comparison was established.
The decision
Simplify the execution path for one bounded task while retaining external verification and mission-level controls. Do not extend this finding to all orchestration or cooperating agents.
What followed
The later coding-team study isolates planner presence on three small code tasks. Its result is a separate experiment on Grex 2.
Evidence and further reading
Public summaries retain the reported counts and limitations. Private execution traces and infrastructure details are not published.
- Download the sanitized evidence summary
Aggregate transcription of retained historical source records. Not a raw-data release or independently reproduced experiment.