Skip to content

Install local speech runtime on demand - #1056

Open
boudra wants to merge 1 commit into
mainfrom
feat-voice-runtime-on-demand
Open

boudra wants to merge 1 commit into
mainfrom
feat-voice-runtime-on-demand

Conversation

@boudra

@boudra boudra commented May 16, 2026

Copy link
Copy Markdown
Collaborator

The daemon no longer bundles the local speech native runtime (sherpa-onnx, sherpa-onnx-node, onnxruntime-node). On first boot, those packages are downloaded into $PASEO_HOME/runtime/local-speech alongside the existing voice model downloads — non-blocking. Users who never enable local voice never pay the size cost.

The wire protocol is unchanged. server_info.capabilities.voice.* already gates the voice UI in the client; readiness now reports runtime_install_in_progress / runtime_missing / runtime_install_failed on the same channel as the existing model-download states. Old clients see voice as unavailable until install completes (same shape as missing models today).

Shape

packages/server/src/server/speech/providers/local/runtime/ is the home for everything install-related (paths, package specs, status, install, package loading, error). External callers see a small surface via the module's index.ts. Provider configs (SherpaOnnxTTS, the recognizers, the VAD session, PocketTtsOnnxTTS) don't know about the install path — the runtime module is configured once at speech-service init and the loaders consult it internally.

The install lifecycle in speech-runtime.ts is one InstallableAsset loop registered with two entries (models, runtime), not two parallel lifecycles.

How to verify beyond CI

  • First-run install: start a fresh daemon with local voice enabled, watch $PASEO_HOME/runtime/local-speech/node_modules/ populate, confirm voice becomes ready without daemon restart, and that server_info.capabilities.voice transitions from the runtime_install_in_progress reason text to ready.
  • Linux + Windows: macOS arm64 is what I exercised. Sherpa ships per-platform packages (sherpa-onnx-linux-x64, sherpa-onnx-win32-x64, etc.) and onnxruntime-node ships per-platform .node binaries. The new bin/napi-*/<platform>/<arch>/*.node validation and the DYLD/LD_LIBRARY_PATH/PATH prepend in sherpa-runtime-env.ts need a real Linux and Windows run before merge.
  • Old client ↔ new daemon: the capability-reason wire format has a stability test (websocket-server.relay-reconnect.test.ts), but a manual run with a 6-month-old client against the new daemon would confirm the new reason codes don't trip parsers.

Risk surface

  • Cross-platform native loading. Only macOS arm64 exercised locally. Loader resolution from $PASEO_HOME plus shared-lib env preparation differs per OS.
  • First-run UX latency. Voice is unavailable until the runtime install completes (hundreds of MB for runtime + models combined). Capability advertisement reports the install state so the client can show "downloading" rather than a generic "unavailable."
  • $PASEO_HOME write access. Install fails if the daemon can't write to the runtime dir. Surfaces as runtime_install_failed with the error string.
  • NPM registry availability. Tarballs are fetched directly from registry.npmjs.org. If the daemon runs offline-first, the existing fallback ("no PASEO_HOME configured → use bundled deps via createRequire") covers dev/test but not production runtime-disabled scenarios.

@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai hello

@paseo-bot

paseo-bot Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Hi! 👋 Smoke test reply from Paseo — everything looks good on my end.

@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai hello

@paseo-bot

paseo-bot Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Hello! 👋 Paseo here — smoke test acknowledged.

@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai rebase this PR

1 similar comment
@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai rebase this PR

@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai what is the most recent commit on main?

@paseo-bot

paseo-bot Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Most recent commit on main:

  • 6dc22483 — fix: update lockfile signatures and Nix hash [skip ci]
  • paseo-ai[bot], 2026-05-21

@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai rebase this PR

The daemon no longer bundles sherpa-onnx, sherpa-onnx-node, or
onnxruntime-node. On first boot they are downloaded into
$PASEO_HOME/runtime/local-speech alongside the existing voice models,
non-blocking. Users who don't enable local voice never pay the size
cost.

The wire protocol is unchanged. server_info.capabilities.voice.*
already gates voice in the client; readiness now reports
runtime_install_in_progress / runtime_missing / runtime_install_failed
on the same channel as the existing model_download_* states.

The native runtime lives under packages/server/src/server/speech/
providers/local/runtime and exposes a small surface: configure the
home once at speech-service init, ensureInstalled, loadPackage,
getStatus, plus a typed error. Provider configs do not know about
the install path.
@boudra
boudra force-pushed the feat-voice-runtime-on-demand branch from e2d8595 to 979326c Compare May 21, 2026 10:46
@boudra

boudra commented May 21, 2026

Copy link
Copy Markdown
Collaborator Author

@paseo-ai what is the most recent commit on main?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant