Grex 2.0 · October–November 2026
Can Grex turn documents into a trustworthy dataset?
A ten-family filing study produced a near-tie with a direct workflow. Corrections added work without improving the final score.
Completed · bounded comparison
Grex passed six of ten document families; the direct workflow passed five. Both matched the identity, date, number and name fields, with remaining acceptance failures concentrated in amendment interpretation. Grex used about 121 aggregate worker minutes versus about 16 for the baseline. This small, model-confounded comparison supports a near-tie, not superiority.
The experiment, explained
A narrated explanation of recorded results, with synthetic narration and visible English captions. Diagrams explain the method; they are not recordings of execution.
Read the video transcript
From documents to a dataset
Can Grex turn public documents into a trustworthy dataset? We compared it with a competent direct workflow on ten filing families, each containing an original document and amendments.
The original question
The study asked whether orchestration and review could improve dependable extraction with less operator work. Procurement access was blocked, so the executed study switched to public securities filings.
The same source material
Both workflows received identical frozen text. Grex used MiniMax; the direct workflow used Claude. A separate blinded evaluator compared outputs with a sealed reference, so model differences remain a limitation.
A near-tie
Grex passed six families and the direct workflow passed five. Remaining failures centered on interpreting amendments. That small difference does not establish a general quality advantage.
Corrections added work
Nine Grex corrections produced no acceptance gain. Aggregate worker time reached about one hundred twenty one minutes, compared with about sixteen for the direct workflow.
Keep the question open
We repaired an ambiguous extraction requirement and tested context efficiency separately. The initial result remains a near-tie, with incomplete evidence about total cost and operator effort.
What were we trying to find out?
Can Grex complete an auditable document-to-dataset task with less operator work than a competent direct workflow?
Hypothesis
Orchestration and independent review might improve dependable extraction while reducing operator work.
Reconstructed from the original prospective study purpose, not a registered numerical prediction. The planned procurement-document corpus was replaced after access was blocked; the executed comparison used public securities filings. Operator effort was not measured well enough to resolve that part of the question.
How the experiment worked
Freeze the same ten filing families for both workflows, produce structured datasets, and score them against a sealed reference with an independent blinded evaluator.
Direct workflow
- Claude worker + tools
- Review and final dataset
Grex
- MiniMax worker + checks
- Independent review + corrections
Blinded reference scoring: direct 5/10 · Grex 6/10
Different worker models; corrections brought no acceptance gain.
Download the method illustration
Sample: Ten public filing families, each combining an original filing and its amendments. Two indeterminate amendment cells were excluded symmetrically after adjudication.
Acquire and repair the corpus
Blocked procurement access led to an approved switch to public securities filings. An extraction defect was repaired before the compared dataset runs.
Freeze identical inputs
Both arms received the same corrected text, schema and document families.
Extract and review
Grex ran one coding mission per family with structural checks and independent review. The direct workflow used ordinary tools and review.
Compare final artifacts
A blinded evaluator scored first-pass and final datasets against a sealed reference; interpretive reference doubts remained.
Controls and differences
- Both arms received identical frozen source material and a common scoring contract.
- Expected answers were sealed from workers; the final evaluator was separate from the producers.
- Worker models differed between arms, and reviewer prompts were not perfectly matched. This was not an isolated test of orchestration.
What happened
| Measure | Direct workflow | Grex |
|---|---|---|
| Final accepted families | 5 / 10 | 6 / 10 |
| Correct / missed / wrong critical cells | 167 / 0 / 5 | 168 / 0 / 4 |
| First-pass accepted families | 5 / 10 | 6 / 10 |
| Aggregate worker time, final | About 16 minutes | About 121 minutes |
| Worker model | Claude Opus 5.5 | MiniMax-M3 |
Corrections did not improve acceptance
Ten Grex root runs and nine corrections still finished at six accepted families. Corrections consumed about 43% of Grex tokens without changing its final accuracy score.
The acquisition path mattered
A blocked procurement source, an incomplete acquisition contract, file-size caps and a filename collision shaped the corpus. The resulting sample favored smaller filings and omitted some intended families.
The contract created avoidable rejection
The schema required exhibit formats that sources rarely stated. That defect caused many first-pass review rejections in both workflows; the later contract allowed an unstated value.
Efficiency became the next question
Grex recorded 41.4 million tokens across the compared work, with roughly 95% of input served from cache. High cache reuse did not remove the large worker-time difference.
What this does not establish
- Ten selected families and different worker models cannot establish a general Grex advantage.
- Amendment interpretation and reference quality affected acceptance. Reference builders, reviewers and evaluator all used Claude-family models, leaving a shared-blind-spot risk.
- Reported worker minutes are aggregate execution time, not elapsed delivery time or active operator effort. Acquisition, setup and review are not fully represented.
- Corpus size limits and acquisition failures restrict coverage; this was not a representative sample of procurement documents or all securities filings.
- Existing subscriptions were used. Full baseline input usage, subscription allocation and total human effort were not measured, so no reliable all-in cost advantage is established.
The decision
Keep the near-tie and the failed acquisition attempts in the record. Repair the extraction contract and test context efficiency separately before claiming scale or lower operating effort.
What followed
The linked context-efficiency study used a different sample and scoring revision. Its denominator is separate from this initial ten-family comparison.
Evidence and further reading
Public summaries retain the reported counts and limitations. Private execution traces and infrastructure details are not published.
- Download the evidence summary
Sanitized aggregate transcription of retained study reports and final closeout evidence. This is not a raw-data release or independent reproduction.