WEBVTT

00:00:00.300 --> 00:00:02.632
Can automatically improved instructions

00:00:02.632 --> 00:00:04.058
produce a better research

00:00:04.058 --> 00:00:05.548
brief than trying again?

00:00:05.848 --> 00:00:06.982
We tested a Harness

00:00:06.982 --> 00:00:08.966
squared candidate around Grex's

00:00:08.966 --> 00:00:10.029
research loop in a

00:00:10.029 --> 00:00:12.013
bounded external experiment.

00:00:12.967 --> 00:00:14.682
That comparison question was

00:00:14.682 --> 00:00:16.465
recorded before execution.

00:00:16.765 --> 00:00:18.811
The reconstructed hypothesis was

00:00:18.811 --> 00:00:20.786
that refined instructions might

00:00:20.786 --> 00:00:22.056
improve the brief.

00:00:22.356 --> 00:00:23.817
The final candidate was

00:00:23.817 --> 00:00:26.153
frozen before held-out evaluation.

00:00:27.133 --> 00:00:28.264
We used one held-out

00:00:28.264 --> 00:00:29.678
group of 3 official

00:00:29.678 --> 00:00:30.739
release pages.

00:00:31.039 --> 00:00:33.334
The conditions shared source material,

00:00:33.334 --> 00:00:34.665
models and criteria.

00:00:34.965 --> 00:00:36.827
An independent evaluator judged

00:00:36.827 --> 00:00:38.422
the outputs without knowing

00:00:38.422 --> 00:00:39.552
their condition.

00:00:40.508 --> 00:00:42.258
Unchanged instructions passed.

00:00:42.558 --> 00:00:43.805
Fresh retries passed on

00:00:43.805 --> 00:00:44.990
the second attempt.

00:00:45.290 --> 00:00:46.992
The refined candidate passed

00:00:46.992 --> 00:00:48.694
mechanical checks but failed

00:00:48.694 --> 00:00:50.328
independent review with 2

00:00:50.328 --> 00:00:51.690
unsupported claims.

00:00:51.990 --> 00:00:53.888
Every attempt remained recorded.

00:00:54.842 --> 00:00:55.438
This is not

00:00:55.438 --> 00:00:56.762
a success-rate study.

00:00:57.062 --> 00:00:58.818
Development adapter defects

00:00:58.818 --> 00:01:00.012
gave the refinement

00:01:00.012 --> 00:01:01.627
process noisy feedback.

00:01:01.927 --> 00:01:03.397
Shared ceilings also did

00:01:03.397 --> 00:01:04.657
not mean equal actual

00:01:04.657 --> 00:01:05.988
spend or complete cost

00:01:05.988 --> 00:01:06.898
accounting.

00:01:07.883 --> 00:01:09.402
We held production adoption

00:01:09.402 --> 00:01:10.857
and stopped expansion from

00:01:10.857 --> 00:01:11.617
this pilot.

00:01:11.917 --> 00:01:13.243
The result rejects this

00:01:13.243 --> 00:01:14.670
candidate on this group,

00:01:14.670 --> 00:01:17.058
not instruction refinement in general.

00:01:17.358 --> 00:01:19.095
A broader transfer experiment

00:01:19.095 --> 00:01:19.897
was not run.