WEBVTT

00:00:00.300 --> 00:00:01.747
Grex implemented a content

00:00:01.747 --> 00:00:03.099
change for WeatherRuta,

00:00:03.099 --> 00:00:04.545
a route weather website.

00:00:04.845 --> 00:00:06.240
The planner page gained

00:00:06.240 --> 00:00:07.565
a road-trip section and

00:00:07.565 --> 00:00:09.517
4 frequently asked questions.

00:00:09.817 --> 00:00:10.966
The change shipped

00:00:10.966 --> 00:00:11.972
on October 30.

00:00:12.925 --> 00:00:14.765
The retained baseline

00:00:14.765 --> 00:00:16.363
reports 19 clicks,

00:00:16.363 --> 00:00:17.962
1198 impressions,

00:00:17.962 --> 00:00:20.286
and average search position

00:00:20.286 --> 00:00:21.352
10 point 6.

00:00:21.652 --> 00:00:23.830
Those numbers are operator-attested

00:00:23.830 --> 00:00:25.446
measurements for October 1

00:00:25.446 --> 00:00:26.430
through 28th.

00:00:27.383 --> 00:00:28.986
The experiment asks whether

00:00:28.986 --> 00:00:29.988
the page reaches a

00:00:29.988 --> 00:00:31.357
position better than 7,

00:00:31.357 --> 00:00:32.759
or doubles its clicks.

00:00:33.059 --> 00:00:34.206
That hypothesis was

00:00:34.206 --> 00:00:35.961
recorded after deployment.

00:00:36.261 --> 00:00:38.991
Observation runs through November 27.

00:00:39.967 --> 00:00:41.020
As of November 4,

00:00:41.020 --> 00:00:41.674
there is no

00:00:41.674 --> 00:00:43.636
completed outcome decision.

00:00:43.936 --> 00:00:45.360
The implementation was

00:00:45.360 --> 00:00:46.819
reviewed and deployed,

00:00:46.819 --> 00:00:47.886
but those facts do

00:00:47.886 --> 00:00:49.310
not establish that more

00:00:49.310 --> 00:00:50.734
people found or visited

00:00:50.734 --> 00:00:51.445
the page.

00:00:52.425 --> 00:00:53.331
This is one page

00:00:53.331 --> 00:00:55.284
without an untreated control.

00:00:55.584 --> 00:00:57.491
The comparison windows differ,

00:00:57.491 --> 00:00:58.601
and a later sitemap

00:00:58.601 --> 00:01:00.960
change also affects interpretation.

00:01:01.260 --> 00:01:02.368
Any movement needs to

00:01:02.368 --> 00:01:03.538
be assessed with those

00:01:03.538 --> 00:01:04.524
limits visible.

00:01:05.508 --> 00:01:06.436
The decision stays

00:01:06.436 --> 00:01:07.770
open until measurement.

00:01:08.070 --> 00:01:09.357
The experiment record

00:01:09.357 --> 00:01:10.712
separates shipped work

00:01:10.712 --> 00:01:12.304
from demonstrated growth,

00:01:12.304 --> 00:01:14.235
and leaves room for improvement,

00:01:14.235 --> 00:01:15.421
no observed effect,

00:01:15.421 --> 00:01:17.115
or an inconclusive result.