Skip to content

Add Pane.capture_since() for cursor-based incremental pane reads - #741

Draft
tony wants to merge 4 commits into
masterfrom
capture-since
Draft

Add Pane.capture_since() for cursor-based incremental pane reads#741
tony wants to merge 4 commits into
masterfrom
capture-since

Conversation

@tony

@tony tony commented Aug 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Add Pane.capture_since() — returns the rows a pane has written since a CaptureCursor, along with a fresh cursor to resume from. Watching a pane over time no longer means re-capturing the whole screen each tick and diffing it by hand.
  • Add libtmux.capture, holding CaptureCursor (immutable, pane-bound, serializable) and the CaptureSince result, split into a pure decision half and a tmux I/O half.
  • Add CaptureCursorError with InvalidCaptureCursor and PaneLifecycleChanged subclasses, so every way a cursor stops being usable is catchable in one except.
  • Report lines_missed and fall back to the visible screen when tmux provably destroyed the anchor, rather than returning a delta that quietly omits rows.
  • Re-anchor by row fingerprint when history-limit trimming makes positional offsets untrustworthy.

Naive screen-diffing goes wrong in three ways tmux makes easy to hit, and a cursor accounts for each: output that scrolled past the visible region between reads; clear-history and history-limit trims renumbering the grid under a stored offset; and a respawned pane reusing its pane_id while running a different program.

Ported from the capture_since tool in tmux-python/libtmux-mcp#63.

Changes by area

Library

  • src/libtmux/capture.py: CaptureCursor, CaptureSince, the cursor codec, anchor-loss and trim-risk arithmetic, fingerprint re-anchoring, and the stable-read driver.
  • src/libtmux/pane.py: Pane.capture_since(), delegating to the module.
  • src/libtmux/exc.py: the three cursor exceptions.

Documentation

  • docs/api/libtmux.capture.md: module reference.
  • docs/topics/pane_interaction.md: a "Capturing only what is new" walkthrough covering the cursor lifecycle, lines_missed recovery, and serialization.

Design decisions

Pure half and I/O half, in one module. A TMUX I/O BOUNDARY marker splits it: above the line, functions decide what a read means given values and run without a tmux server; below it, every function takes a live Pane and issues round-trips. Only the second half is execution-model-specific, so an alternate driver can reuse the anchor arithmetic instead of reimplementing it. It also makes the hardest logic doctestable without a server.

Nothing under src/libtmux/experimental/. That path belongs to the in-flight engine-ops branch; landing there is the one placement that would disturb it.

Truncation stays downstream. max_lines, max_bytes, and the structured truncation metadata remain in libtmux-mcp. Its limiter is a post-filter, and its cursor is built from pane state rather than from the truncated list, so response budgeting cannot corrupt resume semantics. capture_pane returns unbounded lines; capture_since matches it.

The cursor wire format is unchanged. The capture-since-v1: prefix and payload carry across as-is, so cursors already issued downstream keep decoding and that project can drop its own codec.

alternate_on is read but never acted on. It rides along free in the same display-message round-trip. A pane on the alternate screen has handed the whole grid to a full-screen program, so "rows below the anchor" stops carrying delta meaning — but capture-pane -S still returns real main-screen scrollback, so the anchor itself stays arithmetically valid.

Each _cursor_anchor_lost branch is doctested against state where only it fires. A real clear-history trips all three at once, which is enough to prove the function works but not that any individual branch is load-bearing.

The test helper assembles its sentinel from two printf arguments. Polling for a string that also appears in the command line the shell echoes back matches that echo and returns before the payload has run at all.

Coexistence with branches in flight

Measured by rebasing onto engine-seam-minimal and cherry-picking onto engine-ops:

Branch Conflicting files Result
engine-seam-minimal CHANGES Conflicts only in CHANGES; capture and engine suites pass on top
engine-ops CHANGES Conflicts only in CHANGES; capture suite and that branch's ops, docs, and contract suites pass on top

docs/api/index.md initially conflicted as well: engine-seam-minimal inserts its Engine card between Neo and Options. The Capture card sits between Hooks and Constants so both branches auto-merge.

Every tmux round-trip a capture makes goes through Pane.cmd(), which returns self.server.cmd(...), so it dispatches over whatever engine is installed with no adapter. Confirmed by passing a recording SubprocessEngine subclass to Server(engine=...) and observing both capture-pane and display-message arrive at it. Two things to know if you reproduce that: the engine protocol method is run, not execute, and the engine is resolved at construction, so assigning server._engine afterward has no effect.

Verification

Confirm the module holds no dependency on the engine or experimental packages:

$ rg 'libtmux\.(engines|experimental)' src/libtmux/capture.py

Confirm every tmux call site sits below the declared boundary — the marker's line number must be lower than every call site's:

$ rg -n '^# TMUX I/O BOUNDARY|pane\.(cmd|capture_pane|display_message)\(' \
    src/libtmux/capture.py

Confirm the cursor wire format matches the one libtmux-mcp already issues:

$ rg -n 'capture-since-v1' src/libtmux/capture.py

Test plan

  • uv run ruff check . — lint clean
  • uv run ruff format --check . — formatting clean
  • uv run mypy src tests — no type errors
  • uv run pytest tests/test_capture_since.py — delta, anchor-loss, and lifecycle contracts against real tmux
  • uv run pytest --doctest-modules src/libtmux/capture.py — anchor-loss branches isolated, codec round-trip, fingerprint ambiguity refused
  • uv run pytest docs/topics/pane_interaction.md — the documented walkthrough executes
  • just build-docs — builds with no warnings
  • A mutation sweep over the anchor arithmetic, cursor codec, lifecycle guards, and both stability retries — every mutation is caught by the suite
  • Rebased onto engine-seam-minimal and cherry-picked onto engine-ops; both branches' own suites pass with this applied

Two suite failures are pre-existing on master and unrelated: tests/test_server.py::test_new_session_shell_env fails when the shell environment is large enough to exceed tmux's command-length limit, and docs/topics/automation_patterns.md has a doctest whose polling budget is too tight under a slow-starting shell.

Closes #740

@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 62.55924% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.99%. Comparing base (9b0210d) to head (447a162).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/libtmux/capture.py 63.59% 67 Missing and 8 partials ⚠️
src/libtmux/exc.py 0.00% 3 Missing ⚠️
src/libtmux/pane.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #741      +/-   ##
==========================================
+ Coverage   52.37%   52.99%   +0.62%     
==========================================
  Files          26       27       +1     
  Lines        3729     3940     +211     
  Branches      747      783      +36     
==========================================
+ Hits         1953     2088     +135     
- Misses       1472     1540      +68     
- Partials      304      312       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

why: Watching a pane over time meant re-capturing the whole screen each
tick and diffing it by hand. Naive diffing misses output that scrolled
past the visible region, is renumbered by clear-history and
history-limit trims, and reads a respawned pane's new process through a
cursor that still looks valid.

what:
- Add libtmux/capture.py: CaptureCursor, CaptureSince, and the read
  driver, split into a pure decision half and a tmux I/O half
- Add Pane.capture_since(), returning rows written since a cursor plus a
  fresh cursor; the cursor passed in is never advanced
- Fall back to the visible screen with lines_missed when the anchor is
  provably gone, rather than returning an incomplete delta
- Re-locate the anchor by SHA-256 row fingerprint when history-limit
  trimming makes offsets untrustworthy
- Add CaptureCursorError, InvalidCaptureCursor, PaneLifecycleChanged
- Serialize cursors through str()/from_str() for cross-process callers
- Document under docs/api/libtmux.capture.md and pane_interaction.md

Closes #740
@tony

tony commented Aug 15, 2026

Copy link
Copy Markdown
Member Author

Downstream adoption, opened as a draft: tmux-python/libtmux-mcp#122

It pins libtmux to this branch, deletes its own copy of the cursor machinery, and its existing capture_since tests pass unmodified against Pane.capture_since() — the wire format is byte-identical, so previously issued cursors still decode. It stays blocked until this lands and ships in a release.

tony added 3 commits August 15, 2026 06:17
why: The module docstring claimed the pure half ended at
PANE_STATE_FORMAT, but most pure functions sit below that constant, so
the stated boundary did not describe the file. A reader deciding where
to add new logic would have been misled.

what:
- Add an explicit TMUX I/O BOUNDARY marker at the real divide and point
  the module docstring at it instead of a constant
- Remove an unreachable branch in _read_delta: the anchor-loss check
  already proves start is inside the visible region
- Record why _cursor_anchor_lost can ignore shrink magnitude — a larger
  shrink trips the out-of-grid check, and the rest routes to trim risk
- Document that a cursorless call raises PaneLifecycleChanged on an
  already-dead pane, and cover it with a test
why: _raise_if_lifecycle_changed takes a pane id rather than a live
pane and needs no tmux server, so sitting below the I/O boundary
contradicted the marker and hid a piece of decision logic an alternate
driver would want to reuse.

what:
- Move it above the TMUX I/O BOUNDARY, leaving every function below the
  marker one that takes a live Pane
why: Pane.capture_pane returns tmux's stdout without inspecting stderr,
so a failed capture and a blank pane were indistinguishable — the one
case where this module could return an incomplete delta without setting
lines_missed. Separately, the exhausted-retry path took three
unsynchronized reads, pairing an anchor row number with a fingerprint
sampled at a different instant.

what:
- Issue capture-pane directly and raise on stderr; the module only ever
  needs -p, -S and -E, so it does not need the wrapper's flag matrix
- Return the last attempt's paired reads when stability retries are
  exhausted, anchoring cursor and fingerprint to one sample and saving
  three round-trips
- Hash each row once in the fingerprint search instead of once per
  overlapping window
- Make the module-level entry point private; Pane.capture_since and
  CaptureCursor are the public surface
- Add a module logger and a DEBUG line explaining a missed capture
- Correct an inverted bound in the delta comment: start is below
  pane_height, and may be negative to address retained history
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature: upstream capture_since

1 participant