tests(perf[conftest]) Pin $SHELL=/bin/sh β 51% faster suite#1041
Merged
tests(perf[conftest]) Pin $SHELL=/bin/sh β 51% faster suite#1041
Conversation
why: Profiling tmuxp's test suite showed the dominant per-pane cost was tmux spawning the contributor's interactive \$SHELL β typically ``/bin/zsh`` with a full rcfile chain (oh-my-zsh, plugins, prompt themes, etc.) β for every pane created across the suite. Each init costs ~60ms warm and up to ~400ms cold; multiplied across hundreds of pane spawns in workspace/builder tests, that's the single largest line item in suite wall time. tmux falls back to ``\$SHELL`` for new panes when ``default-shell`` is unset (the default in libtmux's bundled test ``.tmux.conf``), so monkey-patching \$SHELL=/bin/sh in an autouse function-scoped fixture forces every test pane to spawn ``/bin/sh`` instead. Empty rcfiles, deterministic prompt, no plugin chains. A/B measured locally with the project's standard ``uv run py.test`` (defaults: ``--reruns=0 --doctest-modules``): master ~76-78s pytest, ~71s wall this branch (1st run) 37.01s pytest, 35.08s wall this branch (2nd run) 36.24s pytest, 34.29s wall That's a ~51% wall-time reduction, consistent with the upstream libtmux PR (tmux-python/libtmux#662) that pins the same env var in libtmux's pytest plugin and reports a similar "~70s -> ~32s" delta. Applying the pin at the tmuxp conftest level lands the gain immediately without waiting for the libtmux PR to merge and release; once libtmux ships the upstream pin, this commit becomes redundant and can be reverted cleanly. what: - conftest.py: add ``_pin_test_shell_env`` function-scoped autouse fixture that calls ``monkeypatch.setenv("SHELL", "/bin/sh")``. ``monkeypatch`` restores the prior value at teardown so no global state leaks across tests or out of pytest's process. - Leading-underscore name signals "internal autouse plumbing" β no test should depend on it explicitly. - USING_ZSH constant and ``zshrc`` autouse-on-USING_ZSH fixture retained for now; they become dead at runtime once \$SHELL is always ``/bin/sh`` (no zsh first-run greeting to suppress) but the cleanup is left as a follow-up to keep this commit's diff minimal and the perf claim independently revertible.
Codecov Reportβ
All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1041 +/- ##
==========================================
+ Coverage 81.96% 81.98% +0.02%
==========================================
Files 28 28
Lines 2545 2548 +3
Branches 485 485
==========================================
+ Hits 2086 2089 +3
Misses 328 328
Partials 131 131 β View full report in Codecov by Sentry. π New features to boost your workflow:
|
why: Document the contributor-side test-suite speedup landing on the unreleased tmuxp 1.68.0 block so anyone skimming next-version notes can confirm the change is dev-only β `tmuxp load`, `tmuxp freeze`, and workspace YAML behaviour are unchanged. what: - Add a Development entry under the unreleased 1.68.0 block, placed after the existing Documentation section per the mandatory section order. Heading style mirrors tmuxp 1.66.0's `#### Structured logging with extra context` entry β name the product surface that changed (test panes), describe what's different (spawn /bin/sh instead of \$SHELL), and cap with a one-line non-user-impact disclaimer. - Frame the speedup as "roughly halve... on systems with slow shell startup" rather than the precise 51% measured locally β the gain depends entirely on the contributor's shell init weight; quoting a single number would be brittle.
Member
Author
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. Claude Code generated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a 14-line autouse fixture to the root
conftest.pythat pins$SHELL=/bin/shfor every test. Dropsuv run py.testwall time by ~51% on this machine β from ~76s to ~36s β by bypassing the contributor's interactive shell init (zsh + oh-my-zsh + plugins, etc.) on every tmux pane the suite spawns.Why this works
tmux falls back to
$SHELLfor new panes when thedefault-shelloption is unset. libtmux's bundled test.tmux.confdoes not setdefault-shell, so every pane in tmuxp's test suite spawns the contributor's interactive$SHELLβ typically/bin/zshβ and pays the full rcfile cost. With hundreds of pane spawns across the workspace/builder tests, that's the single largest line item in suite wall time.monkeypatch.setenv("SHELL", "/bin/sh")in a function-scoped autouse fixture forces every test pane to spawn/bin/shinstead. Empty rcfiles, deterministic prompt, no plugin chains.monkeypatchrestores the prior value at teardown so no global state leaks.Measured timing (this machine,
uv run py.test)Both runs: 797 passed, 2 skipped, zero reruns. Master had occasional reruns on capture-pane assertions that depended on prompt timing; pinning the shell removes that source of flake along with the perf cost.
Relationship to libtmux PR #662
This same fix exists upstream in tmux-python/libtmux#662 (open since 2026-04-29), which pins
$SHELL=/bin/shin libtmux's own pytest plugin and reports a similar "~70s β ~32s" delta on tmuxp's suite. Once libtmux #662 merges and ships in a release tmuxp picks up, this commit becomes redundant and can be reverted cleanly (the autouse fixture would just be a no-op repeating what libtmux's plugin already does).Landing this at the tmuxp conftest level lands the gain immediately rather than waiting on the upstream PR.
What's not in this PR (deliberately)
USING_ZSHconstant andzshrcautouse fixture become dead at runtime once$SHELLis always/bin/sh(no zsh first-run greeting to suppress, no rcfile chain to write a quieting.zshrcfor). They're left in place to keep this PR's diff to the single load-bearing change. Cleanup belongs in a follow-up β easier to review and easier to revert if the perf claim ever needs to be questioned.Test plan
uv run ruff check . --fix --show-fixesβ cleanuv run ruff format .β cleanuv run mypyβ clean (124 source files)uv run py.testβ 797 passed, 2 skipped, no reruns (both runs)just build-docsβ succeeds