Upload log-u8 density grids directly to GPU in reduced-motion mode

[aymuos15]

Reduced-motion density replies decoded the log-u8 wire grid to f32 and re-encoded it with the identical formula before upload. Count-only grids now hand texImage2D the wire bytes unchanged, with pixel-identical output. A copy of the wire bytes is retained on the density object so home-extent restores and context recovery can re-upload without a decoded grid (§27).

CPU removed per density reply, seeded power-law grid, excluding the unchanged gl.texImage2D:

grid	CPU removed	alloc freed
128×96 (12k cells)	0.31 ms (0.28–0.37)	60 KB
256×192 (49k)	1.51 ms (1.47–1.84)	240 KB
512×384 (197k, typical screen)	6.27 ms (5.99–7.04)	960 KB
1024×768 (786k)	24.6 ms (23.6–26.5)	3840 KB
2048×1536 (3.1M, 4K-class)	100.1 ms (97.7–113.5)	15360 KB

Verification

• Under forced reduced motion, every texImage2D payload hashes identically to main in the same order (minus the redundant lodStartNormAnim re-upload), with identical rendered output
• Identity gate: encode(decode(u)) == u for all 256 byte values × 8 max values
• New render smoke probes (dwire, drestore) cover the direct-wire upload and the home-extent restore; both fail against the first commit alone
• All smokes, uv run pytest, Ruff, and pre-commit hooks pass

Measured on: Ubuntu 22.04.5 (Linux 6.8, x86_64), Intel i7-12800H, 30 GB RAM, Node 22.22.2, headless Chromium via SwiftShader.

Benchmark (Node)

// js/src/45_lod.ts, arithmetic verbatim, GL calls removed.

function lodDecodeLogU8(u8, maxVal) {
  const out = new Float32Array(u8.length);
  const denom = Math.log1p(Math.max(0, maxVal || 0));
  if (denom > 0) {
    for (let i = 0; i < u8.length; i++) {
      if (u8[i] > 0) out[i] = Math.expm1((u8[i] / 255) * denom);
    }
  }
  return out;
}

// lodWriteGridTexture, count-only (!rgba) branch.
function encodeR8(f32, maxVal) {
  const denom = Math.log1p(Math.max(0, maxVal || 0));
  const data = new Uint8Array(f32.length);
  if (denom > 0) {
    for (let i = 0; i < f32.length; i++) {
      const c = f32[i];
      if (c > 0 && Number.isFinite(c)) {
        data[i] = Math.max(1, Math.min(255, Math.round(255 * Math.log1p(c) / denom)));
      }
    }
  }
  return data;
}

// Identity gate: the direct path is only sound because decode-then-encode
// reproduces the wire bytes. Algebraically exact, but it is floating point.
const all = new Uint8Array(256);
for (let i = 0; i < 256; i++) all[i] = i;
for (const max of [1, 2, 10, 255, 1e3, 5e5, 1e7, 2 ** 31]) {
  const back = encodeR8(lodDecodeLogU8(all, max), max);
  for (let i = 0; i < 256; i++) {
    if (back[i] !== all[i]) throw new Error(`identity broken: max=${max} byte=${i} -> ${back[i]}`);
  }
}
console.log("identity verified: encode(decode(u)) == u, all 256 bytes x 8 max values\n");

// mulberry32 — reproducible grids.
function rng(seed) {
  let a = seed >>> 0;
  return () => {
    a = (a + 0x6d2b79f5) >>> 0;
    let t = Math.imul(a ^ (a >>> 15), 1 | a);
    t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

// Power-law occupancy: a few hot cells, a sparse tail, mostly empty — the
// shape bin_2d produces, and what drives the branch rate in both loops.
function wireGrid(w, h) {
  const rand = rng(0x5eed);
  const u8 = new Uint8Array(w * h);
  for (let i = 0; i < u8.length; i++) {
    const r = rand();
    if (r < 0.001) u8[i] = 200 + Math.floor(rand() * 55);
    else if (r < 0.05) u8[i] = 40 + Math.floor(rand() * 160);
    else if (r < 0.3) u8[i] = 1 + Math.floor(rand() * 39);
  }
  return u8;
}

const RUNS = 31;
function bench(fn) {
  for (let i = 0; i < 5; i++) fn();
  const t = [];
  for (let r = 0; r < RUNS; r++) {
    const t0 = performance.now();
    fn();
    t.push(performance.now() - t0);
  }
  return t.sort((a, b) => a - b)[(RUNS / 2) | 0];
}

// Reduced motion, count-only: one decode + two encodes (the second from
// lodStartNormAnim's re-upload) vs. the one retained wire-byte copy.
const MAX = 5e5;
for (const [w, h] of [[128, 96], [256, 192], [512, 384], [1024, 768], [2048, 1536]]) {
  const u8 = wireGrid(w, h);
  const before = bench(() => {
    const f32 = lodDecodeLogU8(u8, MAX);
    encodeR8(f32, MAX);
    return encodeR8(f32, MAX);
  });
  const after = bench(() => new Uint8Array(u8));
  const alloc = ((w * h * 5) / 1024).toFixed(0);
  console.log(
    `${String(w + "x" + h).padEnd(11)} ${(w * h / 1000).toFixed(0).padStart(5)}k cells  ` +
    `before ${before.toFixed(3).padStart(8)} ms   after ${after.toFixed(4)} ms   ` +
    `alloc freed ${alloc.padStart(6)} KB`,
  );
}

Run from the repo root: node bench.mjs

Summary by CodeRabbit

Summary by CodeRabbit

• Bug Fixes

  • Improved density rendering for reduced-motion clients when only compact “wire” data is available.
  • Ensured density textures can be restored reliably after switching the displayed area, even if precomputed grid data is missing.

• Tests

  • Added Chromium smoke coverage to verify reduced-motion “wire” handling and correct restoration behavior.

• Documentation

  • Updated the LOD architecture notes to clarify reduced-motion count-only rendering and restore behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 72edb098-daf9-4040-8952-df733c1265d8

📥 Commits

Reviewing files that changed from the base of the PR and between c580e37 and a271f75.

📒 Files selected for processing (1)

• spec/design/lod-architecture.md

---

📝 Walkthrough

Walkthrough

The density texture upload path is refactored into shared helpers. Eligible reduced-motion log-u8 updates without RGBA data now upload raw wire-grid bytes directly, including retained sample-rebin restores, with smoke coverage for both paths.

Changes

Density texture upload flow

Layer / File(s)	Summary
Shared grid upload helpers js/src/45_lod.ts	Grid uploads use a shared byte-upload helper, while lodUploadWireGrid creates R8 textures from raw wire bytes.
Direct density wire upload js/src/45_lod.ts, spec/design/lod-architecture.md	Reduced-motion log-u8 updates without RGBA data retain wire bytes, omit decoded grids, and upload directly; the architecture documentation describes this behavior.
Retained wire rebind and validation js/src/54_kernel.ts, scripts/render_smoke_nonumpy.py	Sample rebinds upload retained wire data when no grid exists, and smoke checks verify initial and restored direct-wire behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

• reflex-dev/xy issue 166 — Implements direct log-u8 wire uploads, including the retained kernel re-bin path.

Suggested reviewers: alek99

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: direct GPU upload of log-u8 density grids for reduced-motion mode.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[codspeed-hq]

Merging this PR will not alter performance

✅ 103 untouched benchmarks
⏩ 2 skipped benchmarks¹

---

_{Comparing aymuos15:perf/density-direct-upload (a271f75) with main (fc09e67)}

Footnotes

1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[aymuos15]

Should I include a change to spec/design/lod-architecture.md as well?

[coderabbitai]

🧹 Nitpick comments (1)

scripts/render_smoke_nonumpy.py (1)

803-808: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert the bytes actually passed to WebGL.

dwire verifies density.wire and texture existence, not the texture upload payload. A regression that decodes/re-encodes—or uploads incorrect bytes—while retaining density.wire still passes. Spy on the relevant texImage2D call (or read back rendered cells) and compare its Uint8Array payload with renc.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/render_smoke_nonumpy.py` around lines 803 - 808, Update the smoke
test around the density upload and `dwire` assertion to capture the relevant
WebGL `texImage2D` call and inspect its `Uint8Array` payload. Compare the bytes
actually submitted to WebGL with `renc`, while retaining the existing wire,
dimensions, normalization, and texture checks.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@scripts/render_smoke_nonumpy.py`:
- Around line 803-808: Update the smoke test around the density upload and
`dwire` assertion to capture the relevant WebGL `texImage2D` call and inspect
its `Uint8Array` payload. Compare the bytes actually submitted to WebGL with
`renc`, while retaining the existing wire, dimensions, normalization, and
texture checks.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 44d727c7-feaa-43ec-af1b-31080d50ca09

📥 Commits

Reviewing files that changed from the base of the PR and between 70c290a and c580e37.

📒 Files selected for processing (3)

• js/src/45_lod.ts
• js/src/54_kernel.ts
• scripts/render_smoke_nonumpy.py

🚧 Files skipped from review as they are similar to previous changes (1)

• js/src/45_lod.ts

[masenf]

Should I include a change to spec/design/lod-architecture.md as well?

yes please. since we are leveraging mostly automated code generation at this point, it's important that the spec and code stays aligned. while we do regularly have an agent comparing the spec documents with the implementation to assure conformance, it makes reviewing individual PRs easier when changes are documented in the spec within the same PR.

and thanks again for your interest and contributions to making this library the best it can be 🚀

[aymuos15]

Thank you very much for explaining in detail, will keep that in mind going forward.