Use dynamic CPU count for -j in vendor/example scripts and docs

[ShamSaleem]

Summary

Follow-up to #20436, which replaced the hardcoded cmake --build -j parallelism in the general
build docs and test/ scripts. This PR finishes the same job for the vendor backend scripts,
example scripts, and their documentation — 65 sites across 37 files, all mechanical:

-j$(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 ))

nproc on Linux, sysctl -n hw.ncpu on macOS (the mps and coreml scripts are Apple-only), and the
arithmetic degrades to -j1 if neither tool exists. "Core count + 1" is the guidance already in
docs/source/using-executorch-building-from-source.md. The pinned values being removed ranged from
-j4 to -j100, including -j64 in the Vulkan test scripts and -j100 in
tools/cmake/preset/README.md.

Two Python sites differ. extension/llm/export/quantizer_lib.py holds a shell command inside a
user-facing error string, so it takes the same shell expression. backends/mlx/test/test_utils.py
builds an argv list handed to subprocess.run with no shell, where a shell expression would reach
cmake as a literal string, so it uses f"-j{(os.cpu_count() or 1) + 1}" instead.

Deliberately out of scope: .ci/** and .github/workflows/**, where the runners are fixed-size and
the parallelism is a resource-tuning decision rather than a portability problem (cuda.yml pins
-j4, likely to bound peak memory); the -j4 in backends/mlx's READMEs and run_all_tests.py,
which is a test-worker count and not a build flag; and docs/source/archive/. Happy to take the CI
files in a separate PR if you'd like them changed.

Review order: the three groups are independent — vendor backend scripts under backends/, example
scripts and docs under examples/, then the two Python files, which are the only sites that are not
a pure token swap.

Partial fix for #10887.

Test plan

ExecuTorch does not build on my Windows host, so verification is static and per-site:

• bash -n passes on all 20 modified shell scripts.
• Every edited command line was re-run with cmake --build/make swapped for echo, confirming all
  65 sites expand to a single valid integer flag (-j17 on this 16-core machine) with zero
  expansion failures. This covers the markdown sites too, including the two lines that begin with
  && and the one with a $ prompt prefix.
• git diff normalised on the -j token shows every removed line has a matching added line, so
  nothing outside the flag changed. Every changed markdown line contains a -j token.
• Both Python files parse; the argv site renders -j17; the instruction string was extracted via
  ast and shell-expanded to confirm a user pasting it gets -j17.
• black --check reports both Python files unchanged. Line endings are unchanged (still LF) and
  git diff --check reports no whitespace errors.
• lintrunner was not run locally — it is not installed on this Windows host and is unavailable
  in my WSL environment, so CI lint is the gate for that. E501 is in the repo's flake8 ignore list,
  so the one long instruction string in quantizer_lib.py (already 165 chars before this change) is
  not a new violation.

This PR was authored with AI assistance (Claude Code); the diff and every verification step above
were reviewed by me.

cc @GregoryComer @digantdesai @cbilgin @JakeStevens @larryliu0820

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21455

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Unrelated Failure

As of commit fee7cc0 with merge base fdae101 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / linux / linux-job (gh)
  pytorch_tokenizers
• pull / unittest-buck / linux / linux-job (gh)
  pytorch_tokenizers
• pull / unittest-editable / linux / linux-job (gh)
  pytorch_tokenizers

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[nil-is-all]

@claude review the PR and the CI failures

[nil-is-all]

@claude review this PR

[claude]

Claude finished @nil-is-all's task in 2m 12s —— View job

---

PR Review

• Read the PR diff (37 files, 65 sites)
• Review shell / markdown / Python changes
• Investigate the CI failures
• Post review

---

Reviewed the full diff against origin/main. This is a clean, well-scoped mechanical change and the substitution is correct across all three groups.

Correctness of the shell expression

-j$(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 )) behaves correctly in every branch:

• Linux — nproc succeeds, sysctl never runs. ✅
• macOS — nproc isn't present (exit 127), stderr suppressed by 2>/dev/null, || falls through to sysctl -n hw.ncpu. The coreml/mps scripts are Apple-only so this is the live path there. ✅
• Neither tool present — inner $(...) expands to empty, so $(( + 1 )) evaluates to 1, giving -j1. ✅

One subtle thing worth calling out as a positive: several of these scripts run under set -e, and a bare $(( expr )) that evaluates to 0 returns exit status 1, which would abort the script. Because the + 1 guarantees the result is always ≥ 1, that trap is avoided. Good that it can never degrade to -j0.

Python sites

• backends/mlx/test/test_utils.py:590 — f"-j{(os.cpu_count() or 1) + 1}" in the argv list is the right call: a shell expression would reach cmake as a literal string via subprocess.run (no shell). os is already imported (line 19), os.cpu_count() can return None so the or 1 guard is correct, and the +1 keeps the semantics consistent with the shell sites. ✅
• extension/llm/export/quantizer_lib.py:112 — this is inside a user-facing instruction string that the user copy-pastes into a shell, so the shell expression is the correct choice here (not the Python form). ✅

Scope

Agree with the exclusions (.ci/**, .github/workflows/**, docs/source/archive/, and the mlx test-worker -j4). CI runners are fixed-size and their -j is a resource-tuning decision, not a portability defect — leaving them pinned is the right judgment.

Nit (non-blocking): the identical 40-char expression is now repeated ~65 times. Nothing to do about it in docs/READMEs, and it faithfully matches the pattern established in #20436, so consistency wins over factoring it into a shared variable in the shell scripts. No change needed.

CI failures — unrelated to this PR

The two pull / unittest failures are not caused by this change. The build succeeds and 1896 tests pass; the job fails at collection on a module unrelated to any file in this diff:

ERROR collecting examples/models/llama3_2_vision/text_decoder/test/test_text_decoder.py
E   ModuleNotFoundError: No module named 'test.test_text_decoder'

That file is not touched by this PR (verified against the diff), and a -j flag swap cannot produce a Python import error. This is a pre-existing test-collection issue on the runner. The third (periodic / gather-models) is flagged by Dr. CI as unclassified because it didn't run on the merge base — also unrelated to a build-parallelism change.

Recommendation: the code change is correct and ready. The red CI is environmental/pre-existing, not introduced here — worth a rebase onto latest main and a re-run to get a green signal, but no code change is required.
· branch fix/10887-portable-cmake-jobs-followup

[nil-is-all]

LGTM, CI failure seem irrelevant. Once @JakeStevens is also happy with it as the approver of the last PR, we are good to merge.

[synath]

Related data point from the Arm Ethos-U getting-started tutorial: it uses -j$(nproc) in its runtime build command, which fails on macOS (no nproc) even though macOS is a supported platform for that flow — sysctl -n hw.ncpu is the Darwin equivalent.

[JakeStevens]

we can merge once conflicts resolved

[ShamSaleem]

Rebased. Only conflicts were the three examples/apple/mps/ files — #22181 removed that backend, so I dropped them. 59 sites / 34 files now.

@synath the Ethos-U one is a bare -j$(nproc), not a hardcoded -jN — different pattern, and there are ~30 of them. Separate PR.

[ShamSaleem]

@JakeStevens rebase is in and the conflicts are gone, but the run came back red on infra rather than the change. The three unittest jobs (linux, editable, buck) all died the same way, in sccache startup before anything compiled:

error sending request for url (http://169.254.169.254/latest/api/token): operation timed out

IMDS timed out so sccache couldn't load AWS credentials, the CMake compiler check failed, and the pytorch_tokenizers wheel build went down with it. The fourth red job is test-qnn-testsuite-linux (qnn, models), which is flaky on main.

Mind kicking off a re-run?