Add on-device custom speech models (transcribe.cpp port from Handy) - #118
Open
ilkerulusoy wants to merge 5 commits into
Open
ilkerulusoy wants to merge 5 commits into
ilkerulusoy wants to merge 5 commits into
Conversation
Owner
|
Can you add a feature to download from Hugging Face maybe? With custom URL and preset values. |
The local speech engine could only use models the user copied in by hand. Add a bundled catalog of ten Whisper-family models and download them directly from Hugging Face, which is the only source: there is no mirror and no bundled copy, so a failed download is reported rather than silently substituted. SpeechModelCatalog.json is compiled into the app, so the model list is complete without network access. Each entry pins an immutable repo revision plus per-quant size and SHA-256, which makes the downloaded bytes provable regardless of CDN routing. ModelDownloadManager streams into `<filename>.partial` and resumes it with a Range header. A 200 to a Range request means the server ignored it, so the partial is truncated instead of having a whole file appended to it; a 206 that starts at the wrong offset discards the partial. Bytes past the catalog size are cut off rather than trusted, and the file is only renamed into place after the SHA-256 matches. Cancel keeps the partial so the next attempt resumes; delete discards it. Settings keeps only the status line and the installed-model picker. Downloading, quant selection, cancel, delete and manual import live in the Manage Models sheet, which stays available while a transfer runs.
Four issues found auditing the download paths: A cancel followed immediately by a restart gave two writers the same `.partial` file handle. `Task.cancel()` only requests a stop, so the first writer was still writing when the second one truncated the file. The SHA-256 check caught the corruption, so no bad model was installed, but the download failed for no reason the user could see. Cancelled tasks are now retained as `retiring` and awaited before the restart, or before a partial delete, touches the file. A finished download did not become the selection, so the first model a user downloaded still needed a separate click to use. It is now selected on completion when nothing usable is selected. A download that failed while the manager sheet was closed left the settings status dot grey with no message. The status line now reports the failure in red. The two "not found" messages pointed at importing a model, which was the only option before this. They now point at Manage Models.
The catalog recorded a license per model but never showed it. Three of the ten models are NVIDIA's under CC-BY-4.0, where attribution is a condition of use rather than a courtesy, so the credit has to be visible wherever the model is offered. Each model card now carries a link to the repo the weights come from, the upstream model they were converted from, and the license, so the card states who made the model before the user downloads it. The sheet header credits the GGUF conversions to handy-computer and transcribe.cpp. `base_model` is kept in the catalog for this, having previously been filtered out when the catalog was generated.
The bundled catalog was the whole list, so a GGUF conversion published by anyone else was unreachable. The model manager now takes a repo id and resolves it against the Hub API, which supplies the commit sha, the file sizes and the LFS SHA-256 for every candidate file. That is the same shape the bundled catalog already has, so an added repo downloads through the same resumable, checksum-verified path with the same guarantee. The sha is pinned when the repo is added, matching the bundled entries: the bytes downloaded later are provably the ones that were inspected. A file is accepted on its first four bytes, not its extension. `.bin` is ambiguous on the Hub, where `openai/whisper-tiny` ships a `pytorch_model.bin` that is a zip archive, so extension alone would have offered a file the engine cannot load. File labels fall back to the full stem when a repo holds several distinct models rather than several quants of one, which is the case for `ggerganov/whisper.cpp`, where quant suffixes repeat across models. Nothing is claimed about an added model's capabilities. The engine reports those when the model loads, so the catalog does not guess.
Author
|
I've add this feature to download models from Hugging Face. area-15-09-2026-15-29.webm |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional Local Model speech engine to Word Tracking. Users can run
custom Whisper-family
.bin/.ggufmodels entirely on-device, with no audioleaving the machine. Apple Speech remains the default engine.
Ported from Handy: the same engine
(transcribe.cpp), the same custom-model discovery idea (drop a
.bin/.ggufinto the models folder and it shows up), and the same language run-plan
semantics.
AI Assistance
This work was done with DeepSeek v4.1 Flash.
Engine / library
TranscribeCpp),consumed as the pinned prebuilt
TranscribeCpp.xcframeworkv0.2.3(SHA-256 verified at build time by SwiftPM). No custom Rust/FFI bridge.
CPU on Intel).
Textream/Vendor/TranscribeCpp(MIT) andlinked only to the macOS target.
How it works
Apple Speech(default) orLocal Model.When Local is selected, the model list lives in the models folder:
Application Support/Textream/Models(sandbox container), scanned for.bin/.gguf. Import… copies a file in; Show Folder / Delete included.transcribe.cpp's native low-latency stream API.
rolling window: re-transcribe every ~1.5 s, 26 s cap, 1 s audio overlap; the
highlight matcher re-anchors when the window rolls.
is used when it matches (
en-US,tr-TR, …), otherwise auto-detect whensupported, otherwise English/first supported language.
engine queue, so the realtime tap never blocks.
close / failure). Pause/resume keeps it loaded.
os_log(
LocalSpeechcategory); the model is unloaded synchronously at terminationto avoid ggml-Metal's static-destructor assertion.
Sample models
Any GGML/GGUF Whisper-family model works. Handy's full catalog lives in
catalog.json(repo/revision/filename/sha256 per model), all mirrored on the
handy-computerHF org.Testing
text for
whisper-tiny.en,whisper-large-v3-turbo,moonshine-streaming-tiny, andnemotron-3.5-asr-streaming(streamingpath), including rapid stop→start.
group only, no package linkage, and the iOS binary contains no local-engine
symbols.
Scope / limitations
so wiring the iOS app is a follow-up.
manually from the links above).
Acknowledgments
its custom-model approach, and Handy's local speech-to-text is built on
transcribe.cpp and ggml.
for cross-platform speech-to-text inference/acceleration.
thanks to the amazing work by Georgi Gerganov and contributors.