SEO finding
Lab runs of the same page differ so much that one run proves little
SEO Info Severity: Low
Part of the audit Page speed and Core Web Vitals. Finding id: seo.labRunsUnstable.
What it costs you
Alerts and fixes based on a single run will chase noise. A real slowdown can hide in the spread.
What to do
Use averages, fix what varies, and make the test see the same version of the page every time.
What it checks
- Speed test results vary too much
How to fix it
Make the runs comparable
- Run the test several times and judge the average, not one run.
- If the site runs A/B tests, set the test to always get one variant (by cookie or user agent).
- If the server sometimes answers with a challenge page or a rate limit, allow the test addresses in the bot protection.
- Check the request lists of the fastest and slowest run for the file that differs.
How to check: Five runs in a row stay within the spread limit.
Thresholds
| Name | Default | Meaning |
|---|---|---|
min_runs | 4 | Starting value. Runs of the same page and device needed before the spread is judged. |
max_spread_pct | 50 | Starting value. Largest LCP minus smallest LCP, as a percentage of the average, before the runs count as unstable. |
How to verify by hand
Run the lab test three more times in a row and compare.
What this relies on
Runs of one page, device and network profile should land within a narrow range.