Real User Monitoring for Ecommerce: RUM vs Synthetic vs CrUX

Performance reports often disagree. Lighthouse says LCP is 1.8s; Search Console says it's 3.2s; the analytics dashboard says median is 2.4s. Each tool is right within its own measurement model — they're measuring different things, in different conditions, on different populations.

For ecommerce stores, understanding the three core performance measurement systems — Chrome User Experience Report (CrUX), Real User Monitoring (RUM), and synthetic testing — is the foundation for an alerting and optimization workflow that actually catches regressions before they affect revenue.

The Three Measurement Systems

1. Chrome User Experience Report (CrUX)

CrUX is Google's public dataset of real Chrome user metrics, aggregated over 28 days from opted-in Chrome installations. It is the dataset Google uses for ranking Core Web Vitals.

Strengths:

• Reflects what Google actually uses for ranking decisions
• No setup required — automatic for any reasonably-trafficked site
• Public — available via API, BigQuery, PageSpeed Insights, Search Console
• Captures real user diversity (devices, networks, geographies)

Limitations:

• 28-day rolling window — slow to surface regressions (a regression today shows up gradually over 28 days)
• Chrome only — no Safari, Firefox, mobile in-app browsers
• Aggregated to URL-pattern level — can't drill down to specific user sessions
• Requires minimum traffic threshold per URL pattern (low-traffic pages have no CrUX data)
• p75 only (the 75th percentile is what's reported, not p50 or p99)

Use CrUX for: confirming what Google sees, ranking-relevant Core Web Vitals tracking, and long-term trend analysis.

2. Real User Monitoring (RUM)

RUM is a JavaScript snippet on every page that captures performance metrics from each real user visit and sends them to a monitoring service. Examples: SpeedCurve LUX, Calibre, DebugBear RUM, Vercel Speed Insights, Datadog RUM, New Relic Browser, Akamai mPulse, Dynatrace RUM.

Strengths:

• Real-time — regressions visible within minutes of deploy
• Full distribution available (p50, p75, p90, p95, p99)
• All browsers, including Safari and Firefox
• Session-level attribution — drill into specific slow sessions
• Custom metrics (cart load time, search latency, checkout step duration)
• Correlates performance with conversion, revenue, and user attributes

Limitations:

• Costs scale with traffic (RUM SaaS pricing: $50-$500+/mo for small stores, $1000s/mo for large)
• Setup overhead — script integration, dashboard configuration, alerting
• Sampling tradeoffs at scale — capturing every session is expensive
• Browser RUM differences (Safari doesn't report all metrics that Chrome does)

Use RUM for: real-time regression detection, conversion correlation, session debugging, and metrics that CrUX doesn't provide (custom timings, business metrics).

3. Synthetic Testing

Synthetic testing runs scripted browser tests against the site from controlled environments (typically AWS or GCP datacenters) on a schedule. Examples: Lighthouse, WebPageTest, SpeedCurve synthetic, Catchpoint, Pingdom, DebugBear, Calibre.

Strengths:

• Reproducible — same network, same device, same timing each run
• Pre-deploy validation — test before a change goes live
• Granular waterfall and timeline data — every request, every event
• Cost predictable (scales with test count, not traffic)
• Can test pages before they have real users (new launches, staging environments)

Limitations:

• Not representative of real user diversity — datacenter networks aren't your customers' networks
• Bot detection — some synthetic tests get fingerprinted and served different content
• Limited concurrency — can't simulate 10,000 concurrent users
• Cache state usually cold — doesn't reflect repeat-visitor experience

Use synthetic for: pre-deploy validation, regression testing in CI, granular debugging, and competitive benchmarking.

When They Disagree

When Lighthouse, CrUX, and RUM disagree, each is usually correct within its measurement model:

• Lighthouse fast, CrUX slow. Synthetic test runs from datacenter on fast network; real users on mobile/4G are slower. RUM data will confirm. The reality is what users experience — fix for that
• RUM fast, CrUX slow. RUM script may load late and miss slow sessions, especially during navigation hops. Or RUM samples Chrome only while CrUX includes all Chrome variants. Check sampling configuration
• CrUX getting worse, RUM stable. CrUX 28-day window is catching up to a regression that happened weeks ago. RUM is showing current state. Check what changed 28+ days ago
• RUM degraded, CrUX stable. A recent deploy regressed performance; CrUX won't catch up for weeks. Trust the RUM signal

The Ecommerce Monitoring Strategy

The pattern that combines the three systems effectively:

1. RUM as the primary alert system. Configure alerts on p75 LCP regression > 200ms over 24 hours, p75 INP regression > 50ms, CLS p75 > 0.1
2. Synthetic as the pre-deploy gate. Lighthouse CI or DebugBear regression tests on staging before promote to prod. Fail builds on regressions > 10%
3. CrUX as the reality check. Weekly review of CrUX trends per template type (homepage, product, collection, checkout). Confirms that real users are experiencing what RUM measures
4. Search Console as the SEO impact monitor. Weekly check of Core Web Vitals report for ranking-relevant changes

Custom Business Metrics

RUM enables custom metrics that the others don't. For ecommerce, the highest-value custom metrics:

• Time to interactive product gallery — when can users actually swipe through product images?
• Add-to-cart click-to-confirm latency — the most conversion-sensitive single interaction on the site
• Search input latency — typing in the search bar; should be sub-50ms or feels broken
• Checkout step transition time — clicking "Continue to shipping" should be sub-1s
• Cart drawer open time — should be near-instant; commonly degraded by app stacking
• Filter/facet apply time — collection page filtering performance

These metrics often matter more for conversion than Core Web Vitals, but only RUM can measure them. They also surface app-stacking and theme regressions that Core Web Vitals miss.

Per-Template Alerting

Different page types have different performance expectations. Aggregate p75 LCP across the whole site hides regressions on specific templates. Configure alerting per template:

• Homepage: LCP target < 2.0s (no excuse — fully owned content)
• Product page: LCP target < 2.5s (hero image dependent)
• Collection page: LCP target < 2.5s (multiple images compete)
• Cart: LCP target < 2.0s (no third-party scripts should be running)
• Checkout: LCP target < 2.0s, INP target < 100ms (highest conversion stakes)
• Blog: LCP target < 3.0s (more lenient for content pages)

The RUM Tool Landscape

RUM tools for ecommerce in 2026:

• SpeedCurve LUX — ecommerce-focused, deep correlation with revenue, mature alerting; $$$$
• Calibre (Vercel-owned) — Lighthouse-based, integrates with Vercel deploys; $$$
• DebugBear — strong on Web Vitals attribution and CSP-friendly RUM; $$
• Vercel Speed Insights — built-in for Vercel-hosted sites, lightweight; included or low cost
• Datadog RUM — best when already using Datadog for backend monitoring; $$$$$ at scale
• New Relic Browser — similar to Datadog, mature enterprise tool; $$$$
• Cloudflare Web Analytics — free, basic Core Web Vitals; lacks alerting depth
• web-vitals.js + custom backend — free, requires building dashboards and alerting yourself

For most independent ecommerce stores, SpeedCurve or DebugBear hits the sweet spot of capability and cost. For Vercel-hosted Next.js storefronts, Vercel Speed Insights is sufficient out-of-the-box.

The Monitoring Checklist

1. RUM script installed on every page, sampling at least 10% of sessions
2. RUM dashboard tracking p75 LCP, INP, CLS per template type
3. Real-time alerts configured on Core Web Vitals regressions > 200ms
4. Synthetic test in CI on staging before every production deploy
5. Lighthouse CI thresholds configured to fail builds on regression
6. CrUX data reviewed weekly per page template
7. Search Console Core Web Vitals report monitored weekly
8. Custom business metrics tracked (add-to-cart, search, checkout step)
9. Performance budget defined per page template
10. Deploy log correlated with RUM regressions to attribute changes

Lab tests catch the bugs you can predict; real user monitoring catches the bugs you can't. For ecommerce stores where performance directly affects revenue, the combination of RUM (primary alerting), synthetic (pre-deploy gates), and CrUX (ranking reality check) is the only configuration that catches regressions fast enough to matter. StoreVitals scans complement RUM by providing synthetic measurements of LCP, INP, and CLS across page templates, integrating Core Web Vitals data from Google's PageSpeed Insights API, and trending the metrics that affect both rankings and conversion.