Skip to content

fix(replay): auto-follow tool activity across tabs - #2029

Merged
Nikhil (shadowfax92) merged 2 commits into
mainfrom
fix/auto-follow-replay-sync
Jul 24, 2026
Merged

Nikhil (shadowfax92) merged 2 commits into
mainfrom
fix/auto-follow-replay-sync

Conversation

@shadowfax92

Copy link
Copy Markdown
Contributor

Problem

Sequential tab chapters (#2024) made the visible tab's rrweb stream be the transport, so any tool that ran after its tab stopped emitting DOM events became unreachable.

In the 2026-07-24 Trendshift session, tab 1 records 9.003s of rrweb events but receives tab-attributed tools at 24.225s, 26.717s and 38.321s. The chapter ended at 9.003s, so those three tools could never become current. Across that session only 6 of 16 tab-attributed tools fell inside their tab's rrweb span at all, and untabbed tools such as name_session were filtered out of every chapter.

This is valid source data — a page can stop mutating while later wait / read / evaluate calls still target it. The picture should freeze; the session should not.

Approach

One global activity clock plus one rrweb player, with the visible tab derived from tool activity rather than accumulated.

  • session-replay.ts (new) — an immutable plan that answers "which tool is current", "which tab should the camera show", and "where is that tab's stream" as a pure function of global time. No camera history, so a backward seek, a forward seek, and a live data refresh all resolve identically. Several actions crossed in one clock tick collapse to the latest playable tab; no intermediate switches are enqueued.
  • usePlayback — owns activity time from animation-frame timestamps instead of mirroring rrweb's clock, which is what stopped dead at the end of a tab. Activity runs from the first playable checkpoint to the later of the last tool and the last rrweb event.
  • ReplayViewport — keeps its single Replayer and aligns it to the projected track time, holding a boundary frame (first checkpoint before, final state after) while global activity continues past the end of the recording. Drift is corrected only when material, not every frame. Caption and address bar now read from separate sources, so an untabbed tool no longer wipes valid tab chrome.
  • Replay.tsx — chapter timers, transitions, completion refs, and pending seeks are deleted rather than adapted. The action timeline is global again: every tool is listed, untabbed and no-visual tools stay current without moving the camera, and clicking a tab pins the camera at the unchanged global time (Resume follow or Play clears it).

Action starts come from the durationMs already on the dispatch row; missing, negative, and non-finite values fall back to completion time deterministically. No backend, schema, generated-DTO, or ingestion changes.

Tests

90 tests across the replay suite, including a fixture reproducing the exact regression shape — a 9.003s track with tools at 2.053s / 24.225s / 26.717s / 38.321s plus an untabbed name_session — asserting every tool stays reachable while the visible track clamps at 9.003s.

Also covered: duration-derived starts and their fallbacks, stable overlap ordering, projection boundaries, deterministic backward/forward seeking, manual pin and resume, live duration growth, one-player lifecycle under Strict Mode, and late player readiness.

Verification

  • bun scripts/run-bun-test.ts apps/claw-app/screens/replay — 90 pass
  • bun scripts/run-bun-test.ts apps/claw-app — full app suite green
  • bun run --filter @browseros/claw-app typecheck — clean
  • bunx biome check apps/claw-app/screens/replay apps/claw-app/modules/api/replay.hooks.ts — clean

Notes

  • The 2× default playback speed on this branch is preserved.
  • The header still shows task wall-clock duration; the transport shows replayable activity. These differ by design.
  • Landed as one commit for the subsystem swap because the plan, the clock, and the viewport change one API together — splitting them would leave intermediate commits that do not compile.

🤖 Generated with Claude Code

Sequential tab chapters (#2024) made the visible tab's rrweb stream the
transport, so any tool that ran after its tab stopped emitting DOM events
became unreachable. In the 2026-07-24 Trendshift session tab 1 records
9.003s of events but receives tools at 24.225s, 26.717s and 38.321s; the
chapter ended at 9.003s and only 6 of 16 tab-attributed tools fell inside
their tab's span at all. Untabbed tools such as name_session were filtered
out of every chapter.

Replaces chapter orchestration with one global activity clock and a pure
session plan:

- session-replay.ts derives the current tool, the visible tab, and the
  tab-local projection as a function of global time. No camera history, so
  a backward seek, a forward seek, and a live refresh all agree.
- usePlayback owns activity time from animation-frame timestamps instead of
  mirroring rrweb, and resumes when a live recording grows past where it
  stopped.
- ReplayViewport keeps its single Replayer and aligns it to the projected
  time, holding a boundary frame while global activity continues past the
  end of the track. Caption and address bar now have separate sources.
- The action timeline is global again: every tool is listed, untabbed and
  no-visual tools stay current without moving the camera, and a tab click
  pins the camera only.

Tool starts come from the durationMs already on the dispatch row; missing,
negative, and non-finite values fall back to completion.
`buildTabView` and `TabView` were re-exported from replay.data.ts for callers
that reached them through that module; the page now builds a session plan
instead, so nothing does. Also refreshes the `startedAtMs` doc for its new
consumer, and keeps a tab selected when a session has no playable recording at
all, matching what the chapter page used to show.
@github-actions github-actions Bot added the fix label Jul 24, 2026
@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Implements a global replay activity clock that keeps tool actions reachable beyond individual tab recordings.

  • Adds an immutable session replay plan for action ordering, tab selection, track projection, and URL resolution.
  • Reworks playback and viewport synchronization around global activity time while clamping rrweb tracks at their boundaries.
  • Adds camera pinning, global action navigation, live-duration growth handling, and extensive replay tests.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete correctness or security defects identified in the changed replay flow.

The global clock, action schedule, tab projection, player alignment, manual pinning, and live-duration behavior are consistently integrated and covered across boundary and regression scenarios.

Important Files Changed

Filename Overview
packages/browseros-agent/apps/claw-app/screens/replay/session-replay.ts Builds the immutable global action schedule, camera state, URL history, and bounded per-tab track projections.
packages/browseros-agent/apps/claw-app/screens/replay/use-playback.ts Replaces rrweb-owned timing with an animation-frame-based global activity clock supporting seeks, speed changes, completion, and live growth.
packages/browseros-agent/apps/claw-app/screens/replay/Replay.tsx Integrates global replay state, automatic tab following, manual camera pinning, transport controls, and the global action timeline.
packages/browseros-agent/apps/claw-app/screens/replay/ReplayViewport.tsx Keeps one rrweb player aligned to projected track time and independently resolves the visible tab chrome and global action caption.
packages/browseros-agent/apps/claw-app/screens/replay/tab-view.ts Refactors tab views into absolute-time rrweb tracks with explicit renderability and recording-boundary metadata.
packages/browseros-agent/apps/claw-app/screens/replay/replay.data.ts Preserves dispatch duration data so the replay plan can derive deterministic action start times.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Data[Replay events and tool dispatches] --> Plan[Session replay plan]
  Clock[Global activity clock] --> Plan
  Plan --> Action[Current tool action]
  Plan --> Camera[Followed or pinned tab]
  Plan --> Projection[Clamped tab track time]
  Camera --> Viewport[Single rrweb viewport]
  Projection --> Viewport
  Action --> Caption[Global action caption and timeline]
Loading

Reviews (1): Last reviewed commit: "refactor(replay): drop re-exports orphan..." | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown
Contributor

✅ Tests passed — 1771/1775

Suite Passed Failed Skipped
✅ agent 356/356 0 0
✅ build 34/34 0 0
✅ claw-app 287/287 0 0
⚠️ claw-mcp 0/0 0 0
⚠️ claw-onboard 0/0 0 0
⚠️ claw-server-rust-quality 0/0 0 0
⚠️ claw-server-rust 0/0 0 0
✅ server-agent 313/313 0 0
✅ server-api 170/170 0 0
✅ server-browser 10/10 0 0
✅ server-integration 10/10 0 0
✅ server-lib 299/300 0 1
✅ server-root 38/41 0 3
✅ server-tools 254/254 0 0

View workflow run

@shadowfax92
Nikhil (shadowfax92) merged commit 9ce27b1 into main Jul 24, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant