Mkulakow/update audio docs - #4400
Conversation
There was a problem hiding this comment.
Pull request overview
Updates OVMS audio-related documentation and demos to reflect current recommended text-to-speech (Kokoro) and expanded speech-to-text examples (Whisper + Qwen ASR) using the OpenAI-compatible /v3/audio/* endpoints.
Changes:
- Switches TTS documentation/examples from SpeechT5 to Kokoro, including
voiceusage in request examples. - Expands the audio demo to include Qwen ASR alongside Whisper and reorganizes sections using tabbed layouts.
- Updates OpenWebUI integration steps to use pre-exported Kokoro models pulled by OVMS.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/parameters.md | Updates CLI parameter documentation for audio (text2speech), including model type guidance. |
| docs/model_server_rest_api_text_to_speech.md | Updates REST example request to Kokoro and adds voice. |
| docs/genai.md | Updates GenAI speech generation blurb to reference Kokoro models. |
| docs/clients_genai.md | Updates client examples (Python + curl) to use Kokoro + voice. |
| demos/integration_with_OpenWebUI/README.md | Revises OpenWebUI demo steps to pull/add a pre-exported Kokoro model. |
| demos/audio/README.md | Reworks audio demo instructions (TTS + STT) with tabs and adds Qwen ASR examples. |
Comments suppressed due to low confidence (4)
demos/audio/README.md:149
- This command is under the CPU heading but no longer sets
--target_device CPU. Because OVMS auto-detects the target device by default, the command may not actually run on CPU on systems with an available GPU.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model dseditor/Qwen3-ASR-1.7B-INT8_OpenVINO --model_name Qwen3-ASR-1.7B-INT8_OpenVINO --model_repository_path /models
demos/audio/README.md:156
- This command is under the GPU heading but no longer sets
--target_device GPU. With the default auto-detection, the actual device selection may not match the heading.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model dseditor/Qwen3-ASR-1.7B-INT8_OpenVINO --model_name Qwen3-ASR-1.7B-INT8_OpenVINO --model_repository_path /models
demos/audio/README.md:572
- This command is labeled CPU but does not set
--target_device CPU. Since OVMS defaults to auto-detection, it may select a different device on systems with a GPU, making the CPU/GPU split misleading.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model OpenVINO/whisper-large-v3-fp16-ov --model_repository_path /models --model_name whisper-large-v3-fp16-ov
demos/audio/README.md:579
- This command is labeled GPU but does not set
--target_device GPU. With the default auto-detection, it may not run on GPU (or may fall back), so the example should be explicit.
docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model OpenVINO/whisper-large-v3-fp16-ov --model_repository_path /models --model_name whisper-large-v3-fp16-ov
| When pulling models outside of OpenVINO organization the optimum-cli api is used inside ovms. You can set additional parameters for this mode. | ||
| | Option | Value format | Description | | ||
| |------------------------------|--------------|---------------------------------------------------------------------------------------------------------------| | ||
| | `--extra_quantization_params`| `string` | Add advanced quantization parameters. Check [optimum-intel](https://github.com/huggingface/optimum-intel) documentation. Example: `--sym --group-size -1 --ratio 1.0 --awq --scale-estimation --dataset wikitext2` | | ||
| | `--weight-format` | `string` | Model precision used in optimum-cli export with conversion. Default `int8`. | | ||
| | `--vocoder` | `string` | The vocoder model to use for text2speech. For example `microsoft/speecht5_hifigan`. | | ||
|
|
| ### Text to speech | ||
| | option | Value format | Description | | ||
| |---------------------------|--------------|--------------------------------------------------------------------------------| | ||
| | `--num_streams` | `integer` | The number of parallel execution streams to use for the model. Use at least 2 on 2 socket CPU systems. Default: 1. | | ||
| | `--model_type` | `string` | Type of the source TTS model: `speecht5` (default) or `kokoro`. | | ||
| | `--vocoder` | `string` | The vocoder model to use for text2speech. For example `microsoft/speecht5_hifigan`. | | ||
| | `--model_type` | `string` | Type of the source TTS model. Supported values: `kokoro` (default, recommended) and `speecht5` (legacy). | | ||
|
|
| ```bash | ||
| mkdir -p models | ||
| docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --task speech2text --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models --target_device CPU | ||
| docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v $(pwd)/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models |
| ```bash | ||
| mkdir -p models | ||
| docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --task speech2text --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models --target_device GPU | ||
| docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model OpenVINO/whisper-large-v3-turbo-fp16-ov --model_name whisper-large-v3-turbo-fp16-ov --model_repository_path /models |
1f73c03 to
b29bbca
Compare
b29bbca to
9fbfb2a
Compare
| dataclasses==0.6 | ||
| transformers<5.1 | ||
| numpy==2.3.5 | ||
| numpy==2.2.6 |
There was a problem hiding this comment.
whats the reason of numpy downgrade?
There was a problem hiding this comment.
CVS-185637
E Code: 1
E Stderr: ERROR: Ignored the following versions that require a different python version: 0.7 Requires-Python >=3.6,<3.7; 0.8 Requires-Python >=3.6,<3.7; 2.3.0 Requires-Python >=3.11; 2.3.1 Requires-Python >=3.11; 2.3.2 Requires-Python >=3.11; 2.3.3 Requires-Python >=3.11; 2.3.4 Requires-Python >=3.11; 2.3.5 Requires-Python >=3.11; 2.4.0 Requires-Python >=3.11; 2.4.0rc1 Requires-Python >=3.11; 2.4.1 Requires-Python >=3.11; 2.4.2 Requires-Python >=3.11; 2.4.3 Requires-Python >=3.11; 2.4.4 Requires-Python >=3.11
E ERROR: Could not find a version that satisfies the requirement numpy==2.3.5 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4, 1.25.0, 1.25.1, 1.25.2, 1.26.0, 1.26.1, 1.26.2, 1.26.3, 1.26.4, 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 2.2.6)
E ERROR: No matching distribution found for numpy==2.3.5
| docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 -v ${HOME}/models:/models:rw openvino/model_server:latest --rest_port 8000 --source_model luis-castillo/Kokoro-82M-OpenVINO-FP16-OVMS --model_repository_path /models --model_name Kokoro-82M-OpenVINO-FP16-OVMS --target_device CPU | ||
| ``` | ||
|
|
||
| **Deploying on Bare Metal** | ||
|
|
||
| ```bat | ||
| mkdir c:\models | ||
| ovms --rest_port 8000 --source_model luis-castillo/Kokoro-82M-OpenVINO-FP16-OVMS --model_repository_path c:\models --model_name Kokoro-82M-OpenVINO-FP16-OVMS --target_device CPU --task text2speech | ||
| ovms --rest_port 8000 --source_model luis-castillo/Kokoro-82M-OpenVINO-FP16-OVMS --model_repository_path c:\models --model_name Kokoro-82M-OpenVINO-FP16-OVMS --target_device CPU |
There was a problem hiding this comment.
target device was removed in other places - should it still be here?
There was a problem hiding this comment.
yes, since kokoro does not work well on gpu
| ```bash | ||
| mkdir -p models | ||
| docker run -d -u $(id -u):$(id -g) --rm -p 8000:8000 --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) -v $(pwd)/models:/models:rw openvino/model_server:latest-gpu --rest_port 8000 --source_model luis-castillo/Kokoro-82M-OpenVINO-FP16-OVMS --model_repository_path /models --model_name Kokoro-82M-OpenVINO-FP16-OVMS --target_device GPU --task text2speech | ||
| docker run -d ${GPU_ARGS} -u $(id -u):$(id -g) --rm -p 8000:8000 -v ${HOME}/models:/models:rw openvino/model_server:weekly --rest_port 8000 --source_model luis-castillo/Kokoro-82M-OpenVINO-FP16-OVMS --model_repository_path /models --model_name Kokoro-82M-OpenVINO-FP16-OVMS |
There was a problem hiding this comment.
If Kokoro works badly with GPU does it make sense to include that part where sort of default option is GPU?
🛠 Summary
JIRA/Issue if applicable.
Describe the changes.
🧪 Checklist
``