demo: Add benchmark-react with normalization and ref-stability scenarios by ntucker · Pull Request #3783 · reactive/data-client

ntucker · 2026-03-09T03:29:24Z

Motivation

We need a browser-based benchmark for @data-client/react that highlights the performance advantages of normalization and referential equality: when one entity is updated, only components that use that entity get new references; others keep the same object reference so React can skip rerenders. This enables comparison against other React data libraries (e.g. TanStack Query, SWR) and surfaces regressions in CI.

Solution

New example examples/benchmark-react: Playwright-driven benchmark that runs a small React app with shared authors, measures mount/update duration via performance.measure(), and reports in customSmallerIsBetter format for rhysd/github-action-benchmark.
CI runs data-client only: Hot-path scenarios for data-client track our regressions; competitor libraries (TanStack Query, SWR, baseline) are benchmarked locally for comparison only.
React Compiler support: yarn bench:run:compiler builds with babel-plugin-react-compiler (via @anansi/babel-preset's reactCompiler option) and labels results [compiler] for side-by-side comparison in the report viewer.

Scenarios

Hot path (CI, data-client only)

Mount (100/500 items), update single entity, update shared author (duration + scaling to 500/1000 mounted)
Ref-stability (count of components receiving new object references — smaller is better)
Sorted/derived view (Query schema memoization vs useMemo sort)
Bulk ingest (500 items through normalization pipeline)
Optimistic update (data-client only)
Invalidate and resolve (data-client only — Suspense boundary round-trip)

Local comparison only

With-network: simulated delay per request (data-client = 1 update vs N refetches for non-normalized libs)
Memory: heap delta after mount/unmount cycles
Startup: FCP and task duration via CDP

Key design decisions

4x CPU throttling via CDP to amplify small differences on fast machines
Interleaved library execution with shuffled order per round to reduce environmental variance
Three measurement layers: performance.measure() (JS-driven), React Profiler actualDuration (reconciliation), Chrome trace duration (full rendering pipeline)
Warmup runs discarded; report median with 95% CI
Report viewer (bench/report-viewer.html): filterable table (base/react-commit/trace), time-series charting via history file loading

Scripts

Script	Description
`yarn bench:run`	Build + serve + bench (all libraries locally)
`yarn bench:run:compiler`	Same but with React Compiler enabled, results labelled `[compiler]`
`yarn build:compiler`	Build with React Compiler only
`yarn bench:compiler`	Run bench with `[compiler]` label only

Open questions

N/A

Made with Cursor

Note

Medium Risk
Adds a sizable new benchmark app plus Playwright-based CI workflow changes, which can impact CI runtime/stability and benchmark reporting despite not affecting production library code paths.

Overview
Adds a new browser-driven benchmark suite examples/benchmark-react to measure React rendering performance and ref-stability across @data-client/react and competitor libraries, including a Playwright runner, scenario definitions, and a local report viewer.

Introduces a new GitHub Actions workflow benchmark-react.yml to run these benchmarks on relevant PRs/pushes (CI executes data-client-only hot-path scenarios) and updates the existing Node benchmark workflow with permissions/concurrency plus a cache-key fix. Also wires the new workspace + build:benchmark-react script into the repo, updates benchmark documentation/agent guidance, and refreshes playground RestEndpoint editor type definitions.

^{Written by Cursor Bugbot for commit 05e8319. This will update automatically on new commits. Configure here.}

vercel · 2026-03-09T03:29:29Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs-site	Ignored	Preview	Mar 11, 2026 1:45pm

changeset-bot · 2026-03-09T03:29:32Z

⚠️ No Changeset found

Latest commit: 05e8319

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-03-09T03:33:59Z

Size Change: 0 B

Total Size: 80.5 kB

ℹ️ View Unchanged

Filename	Size
`examples/test-bundlesize/dist/App.js`	3.18 kB
`examples/test-bundlesize/dist/polyfill.js`	307 B
`examples/test-bundlesize/dist/rdcClient.js`	10.2 kB
`examples/test-bundlesize/dist/rdcEndpoint.js`	6.33 kB
`examples/test-bundlesize/dist/react.js`	59.7 kB
`examples/test-bundlesize/dist/webpack-runtime.js`	726 B

_{compressed-size-action}

codecov · 2026-03-09T03:36:39Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.05%. Comparing base (3b85c82) to head (05e8319).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3783   +/-   ##
=======================================
  Coverage   98.05%   98.05%           
=======================================
  Files         151      151           
  Lines        2834     2834           
  Branches      555      555           
=======================================
  Hits         2779     2779           
  Misses         11       11           
  Partials       44       44

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

examples/benchmark-react/bench/stats.ts

cursor · 2026-03-09T03:37:40Z

examples/benchmark-react/bench/tracing.ts

+        e.name === 'FunctionCall' ||
+        e.name?.includes('EvaluateScript') ||
+        e.cat?.includes('devtools.timeline'),
+    );


Variable scriptEvents computed but never used

Low Severity

The scriptEvents variable in parseTraceDuration is computed by filtering events but is never read or referenced anywhere afterward. This is dead code that adds unnecessary computation.

examples/benchmark-react/bench/tracing.ts

examples/benchmark-react/src/data-client/resources.ts

examples/benchmark-react/bench/runner.ts

examples/benchmark-react/bench/scenarios.ts

examples/benchmark-react/bench/tracing.ts

examples/benchmark-react/package.json

examples/benchmark-react/bench/runner.ts

examples/benchmark-react/src/data-client/index.tsx

examples/benchmark-react/bench/runner.ts

- Browser benchmark comparing @data-client/react (Playwright, customSmallerIsBetter). - Scenarios: mount, update entity/author, ref-stability (item/author ref counts). - Hot-path (CI) vs with-network (local): simulated delay for overfetch comparison. - CI workflow runs hot-path only; reports to rhysd/github-action-benchmark. Made-with: Cursor

cursor · 2026-03-10T13:49:15Z

examples/benchmark-react/src/swr/index.tsx

+                prev ? { ...prev, author: newAuthor } : prev,
+              );
+            }
+          }


SWR/TanStack updateAuthor iterates all 1000 fixture items

Low Severity

The updateAuthor callbacks in both SWR and TanStack Query implementations loop over all 1000 FIXTURE_ITEMS to find items matching the author, calling mutate/setQueryData for ~50 items (1000 items / 20 authors). However, only 100 items are typically mounted, so ~45 of those cache writes target unmounted items. This unnecessary work inflates the update-shared-author-duration measurement for these libraries compared to what a real-world implementation would do.

Additional Locations (1)

examples/benchmark-react/src/tanstack-query/index.tsx#L142-L149

github-actions

Benchmark

Details

Benchmark suite	Current: `05e8319`	Previous: `3b85c82`	Ratio
`normalizeLong`	`441` ops/sec (`±0.95%`)	`458` ops/sec (`±0.70%`)	`1.04`
`normalizeLong Values`	`404` ops/sec (`±0.35%`)	`414` ops/sec (`±0.45%`)	`1.02`
`denormalizeLong`	`282` ops/sec (`±2.42%`)	`299` ops/sec (`±2.23%`)	`1.06`
`denormalizeLong Values`	`261` ops/sec (`±2.07%`)	`267` ops/sec (`±2.69%`)	`1.02`
`denormalizeLong donotcache`	`1041` ops/sec (`±0.16%`)	`1052` ops/sec (`±0.15%`)	`1.01`
`denormalizeLong Values donotcache`	`766` ops/sec (`±0.22%`)	`772` ops/sec (`±0.50%`)	`1.01`
`denormalizeShort donotcache 500x`	`1591` ops/sec (`±0.15%`)	`1434` ops/sec (`±0.15%`)	`0.90`
`denormalizeShort 500x`	`854` ops/sec (`±2.24%`)	`792` ops/sec (`±2.08%`)	`0.93`
`denormalizeShort 500x withCache`	`6211` ops/sec (`±0.11%`)	`5211` ops/sec (`±0.57%`)	`0.84`
`queryShort 500x withCache`	`2715` ops/sec (`±0.12%`)	`2648` ops/sec (`±0.22%`)	`0.98`
`buildQueryKey All`	`54065` ops/sec (`±0.47%`)	`56000` ops/sec (`±0.51%`)	`1.04`
`query All withCache`	`7106` ops/sec (`±0.16%`)	`6060` ops/sec (`±0.49%`)	`0.85`
`denormalizeLong with mixin Entity`	`272` ops/sec (`±2.28%`)	`284` ops/sec (`±2.20%`)	`1.04`
`denormalizeLong withCache`	`5949` ops/sec (`±0.12%`)	`7079` ops/sec (`±0.31%`)	`1.19`
`denormalizeLong Values withCache`	`5057` ops/sec (`±0.29%`)	`4556` ops/sec (`±0.19%`)	`0.90`
`denormalizeLong All withCache`	`7014` ops/sec (`±0.32%`)	`4980` ops/sec (`±0.14%`)	`0.71`
`denormalizeLong Query-sorted withCache`	`7110` ops/sec (`±0.22%`)	`5793` ops/sec (`±0.13%`)	`0.81`
`denormalizeLongAndShort withEntityCacheOnly`	`1666` ops/sec (`±0.42%`)	`1534` ops/sec (`±0.48%`)	`0.92`
`getResponse`	`4602` ops/sec (`±0.56%`)	`3652` ops/sec (`±0.58%`)	`0.79`
`getResponse (null)`	`9986046` ops/sec (`±0.93%`)	`9869619` ops/sec (`±0.70%`)	`0.99`
`getResponse (clear cache)`	`258` ops/sec (`±2.22%`)	`276` ops/sec (`±2.36%`)	`1.07`
`getSmallResponse`	`3385` ops/sec (`±0.11%`)	`3347` ops/sec (`±0.47%`)	`0.99`
`getSmallInferredResponse`	`2499` ops/sec (`±0.12%`)	`2566` ops/sec (`±0.23%`)	`1.03`
`getResponse Collection`	`4570` ops/sec (`±0.70%`)	`3722` ops/sec (`±0.09%`)	`0.81`
`get Collection`	`4556` ops/sec (`±0.23%`)	`3330` ops/sec (`±0.18%`)	`0.73`
`get Query-sorted`	`5170` ops/sec (`±0.40%`)	`4875` ops/sec (`±0.48%`)	`0.94`
`setLong`	`438` ops/sec (`±0.38%`)	`462` ops/sec (`±0.25%`)	`1.05`
`setLongWithMerge`	`255` ops/sec (`±0.41%`)	`260` ops/sec (`±0.23%`)	`1.02`
`setLongWithSimpleMerge`	`267` ops/sec (`±0.39%`)	`278` ops/sec (`±0.45%`)	`1.04`
`setSmallResponse 500x`	`947` ops/sec (`±0.10%`)	`922` ops/sec (`±0.49%`)	`0.97`

This comment was automatically generated by workflow using github-action-benchmark.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

cursor · 2026-03-11T13:58:35Z

examples/benchmark-react/src/swr/index.tsx

+
+const rootEl = document.getElementById('root') ?? document.body;
+createRoot(rootEl).render(
+  <SWRConfig value={{ provider: () => cache as any }}>


SWR app missing revalidation disable skews benchmark results

Medium Severity

The SWR app's SWRConfig doesn't disable automatic revalidation, unlike the TanStack Query app which sets staleTime: Infinity and gcTime: Infinity. SWR's default behavior triggers a background fetch on every useSWR mount. Since fetcher always rejects, each of the 100–1000 mounted ItemView components triggers an unnecessary Promise.reject, error handling, and potential extra re-render. This systematically inflates SWR's measured durations compared to the other libraries, producing misleading local comparison results.

ntucker force-pushed the benchmark-react-normalization branch from af004f6 to a2cb25d Compare March 9, 2026 03:32

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/scenarios.ts Outdated Show resolved Hide resolved

cursor bot reviewed Mar 9, 2026

View reviewed changes

examples/benchmark-react/bench/tracing.ts Outdated Show resolved Hide resolved

examples/benchmark-react/package.json Outdated Show resolved Hide resolved

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Show resolved Hide resolved

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/src/data-client/index.tsx Show resolved Hide resolved

cursor bot reviewed Mar 10, 2026

View reviewed changes

examples/benchmark-react/bench/runner.ts Outdated Show resolved Hide resolved

ntucker added 12 commits March 10, 2026 09:21

more of the plan

668b29f

More scenarios

c4c4ac9

more scenarios

e5a92b6

add react compiler option

1c4278d

bugbot: dead code

08acdb8

No throttling

4b81222

yarn lock

6651c08

bugbot + fix test data client correctness

0ab9478

fair comparisons

c2d97cc

ts 6

8fbf916

bugbot

98de6fa

ntucker force-pushed the benchmark-react-normalization branch from d8d904f to 98de6fa Compare March 10, 2026 13:31

update website types

22daa22

vercel bot deployed to Preview March 10, 2026 13:36 View deployment

cursor bot reviewed Mar 10, 2026

View reviewed changes

improve ci

48dea4d

github-actions bot reviewed Mar 10, 2026

View reviewed changes

internal: Bench runs concurrently

05e8319

cursor bot reviewed Mar 11, 2026

View reviewed changes

Uh oh!

Conversation

ntucker commented Mar 9, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Solution

Scenarios

Key design decisions

Scripts

Open questions

Uh oh!

vercel bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

cursor bot Mar 9, 2026

Choose a reason for hiding this comment

Variable scriptEvents computed but never used

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor bot Mar 10, 2026

Choose a reason for hiding this comment

SWR/TanStack updateAuthor iterates all 1000 fixture items

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Mar 11, 2026

Choose a reason for hiding this comment

SWR app missing revalidation disable skews benchmark results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ntucker commented Mar 9, 2026 •

edited by cursor bot

Loading

vercel bot commented Mar 9, 2026 •

edited

Loading

changeset-bot bot commented Mar 9, 2026 •

edited

Loading

github-actions bot commented Mar 9, 2026 •

edited

Loading

codecov bot commented Mar 9, 2026 •

edited

Loading

Variable `scriptEvents` computed but never used

SWR/TanStack `updateAuthor` iterates all 1000 fixture items

github-actions bot left a comment •

edited

Loading