Skip to content

Commit d04ee6c

Browse files
LauraGPTLauraGPT
authored
Merge pull request #3577 from modelscope/codex/moss-sglang-product-site
docs: add native SGLang path for MOSS diarization
2 parents 8d65a38 + af474f3 commit d04ee6c

15 files changed

Lines changed: 261 additions & 23 deletions

‎docs/community_projects.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ These integrations are community-maintained. Their release cadence, hardware sup
4343
| [OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) | A multi-agent interactive classroom with a built-in local FunASR provider. It sends WAV audio to the OpenAI-compatible transcription endpoint, requires no API key, and exposes SenseVoiceSmall, Paraformer, and Fun-ASR-Nano through its model selector. | [FunASR local ASR setup](https://github.com/THU-MAIC/OpenMAIC#funasr-local-asr), configure `ASR_FUNASR_BASE_URL`, and see merged provider [#1044](https://github.com/THU-MAIC/OpenMAIC/pull/1044). |
4444
| [Optimum Intel](https://github.com/huggingface/optimum-intel) | OpenVINO export and inference support for Fun-ASR models through Hugging Face Optimum Intel, including FunASR-specific modeling and ASR/export/quantization coverage. | [FunASR OpenVINO modeling](https://github.com/huggingface/optimum-intel/blob/main/optimum/intel/openvino/modeling_funasr.py), [supported models docs](https://github.com/huggingface/optimum-intel/blob/main/docs/source/openvino/models.mdx), and merged [#1801](https://github.com/huggingface/optimum-intel/pull/1801). |
4545
| [OpenVINO Notebooks](https://github.com/openvinotoolkit/openvino_notebooks) | Official OpenVINO tutorial material for FunASR-Nano, including helper code and NPU device fixes for running the notebook on Intel acceleration paths. | [FunASR-Nano notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/funasr-nano), [OpenVINO FunASR helper](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/funasr-nano/ov_funasr_helper.py), merged [#3497](https://github.com/openvinotoolkit/openvino_notebooks/pull/3497), and NPU fix [#3517](https://github.com/openvinotoolkit/openvino_notebooks/pull/3517). |
46-
| [SGLang Omni](https://github.com/sgl-project/sglang-omni) | A multi-stage pipeline runtime for omni models with built-in Fun-ASR model support, OpenAI-compatible serving, cookbook docs, benchmark tasks, and formatted stream-output tests. | [Fun-ASR cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/fun_asr.md), [Fun-ASR model runtime](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/fun_asr), merged [#1078](https://github.com/sgl-project/sglang-omni/pull/1078), and merged review-fix [#1079](https://github.com/sgl-project/sglang-omni/pull/1079). |
46+
| [SGLang Omni](https://github.com/sgl-project/sglang-omni) | A multi-stage pipeline runtime for omni models with built-in Fun-ASR and MOSS-Transcribe-Diarize support, OpenAI-compatible serving, cookbook docs, benchmark tasks, and formatted output tests. | [Fun-ASR cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/fun_asr.md), [Fun-ASR model runtime](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/fun_asr), merged Fun-ASR [#1078](https://github.com/sgl-project/sglang-omni/pull/1078) and [#1079](https://github.com/sgl-project/sglang-omni/pull/1079), plus merged native MOSS pipeline and H100 validation [#914](https://github.com/sgl-project/sglang-omni/pull/914). |
4747
| [Fun-ASR-vLLM](https://github.com/yuekaizhang/Fun-ASR-vllm) | Community vLLM inference for Fun-ASR-Nano and Fun-ASR-MLT-Nano, including batch evaluation and NVIDIA Triton deployment. | [Setup and benchmarks](https://github.com/yuekaizhang/Fun-ASR-vllm#readme) and the deterministic ASR decoding fix in merged [#20](https://github.com/yuekaizhang/Fun-ASR-vllm/pull/20). |
4848
| [vad-burn](https://github.com/di-osc/vad-burn) | FSMN VAD inference in pure Rust with Python bindings, including offline, streaming, and CPU-only modes. | [Project README](https://github.com/di-osc/vad-burn#readme) and the FunASR showcase in [#3106](https://github.com/modelscope/FunASR/issues/3106). |
4949

‎docs/community_projects_zh.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@
4343
| [OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) | 多智能体互动课堂，已内置本地 FunASR provider。它把 WAV 音频发送到 OpenAI 兼容转写端点，无需 API Key，并可在模型选择器中使用 SenseVoiceSmall、Paraformer 和 Fun-ASR-Nano。 | [FunASR 本地 ASR 配置](https://github.com/THU-MAIC/OpenMAIC#funasr-local-asr)，设置 `ASR_FUNASR_BASE_URL`，并参见已合并 provider [#1044](https://github.com/THU-MAIC/OpenMAIC/pull/1044)。 |
4444
| [Optimum Intel](https://github.com/huggingface/optimum-intel) | 通过 Hugging Face Optimum Intel 支持 Fun-ASR 模型的 OpenVINO 导出与推理，包括 FunASR 专用 modeling 代码以及 ASR、export、quantization 覆盖。 | [FunASR OpenVINO modeling](https://github.com/huggingface/optimum-intel/blob/main/optimum/intel/openvino/modeling_funasr.py)、[支持模型文档](https://github.com/huggingface/optimum-intel/blob/main/docs/source/openvino/models.mdx) 和已合并 [#1801](https://github.com/huggingface/optimum-intel/pull/1801)。 |
4545
| [OpenVINO Notebooks](https://github.com/openvinotoolkit/openvino_notebooks) | OpenVINO 官方 FunASR-Nano 教程资料，包含 helper 代码和 NPU device 修复，方便用户在 Intel 加速路径上运行 notebook。 | [FunASR-Nano notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/funasr-nano)、[OpenVINO FunASR helper](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/funasr-nano/ov_funasr_helper.py)、已合并 [#3497](https://github.com/openvinotoolkit/openvino_notebooks/pull/3497) 和 NPU 修复 [#3517](https://github.com/openvinotoolkit/openvino_notebooks/pull/3517)。 |
46-
| [SGLang Omni](https://github.com/sgl-project/sglang-omni) | 面向 omni models 的 multi-stage pipeline runtime，已内置 Fun-ASR 模型支持、OpenAI 兼容服务、cookbook 文档、benchmark 任务和格式化后的 stream-output 测试。 | [Fun-ASR cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/fun_asr.md)、[Fun-ASR 模型 runtime](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/fun_asr)、已合并 [#1078](https://github.com/sgl-project/sglang-omni/pull/1078) 和 review-fix 已合并 [#1079](https://github.com/sgl-project/sglang-omni/pull/1079)。 |
46+
| [SGLang Omni](https://github.com/sgl-project/sglang-omni) | 面向 omni models 的 multi-stage pipeline runtime，已内置 Fun-ASR 与 MOSS-Transcribe-Diarize 支持、OpenAI 兼容服务、cookbook 文档、benchmark 任务和格式化输出测试。 | [Fun-ASR cookbook](https://github.com/sgl-project/sglang-omni/blob/main/docs/cookbook/fun_asr.md)、[Fun-ASR 模型 runtime](https://github.com/sgl-project/sglang-omni/tree/main/sglang_omni/models/fun_asr)、已合并 Fun-ASR [#1078](https://github.com/sgl-project/sglang-omni/pull/1078) 和 [#1079](https://github.com/sgl-project/sglang-omni/pull/1079)，以及已合并的原生 MOSS pipeline 与 H100 验证 [#914](https://github.com/sgl-project/sglang-omni/pull/914)。 |
4747
| [Fun-ASR-vLLM](https://github.com/yuekaizhang/Fun-ASR-vllm) | 面向 Fun-ASR-Nano 和 Fun-ASR-MLT-Nano 的社区 vLLM 推理实现，包含批量评测和 NVIDIA Triton 部署。 | [安装与 Benchmark](https://github.com/yuekaizhang/Fun-ASR-vllm#readme) 和已合并 [#20](https://github.com/yuekaizhang/Fun-ASR-vllm/pull/20) 中的确定性 ASR 解码修复。 |
4848
| [vad-burn](https://github.com/di-osc/vad-burn) | 纯 Rust FSMN VAD 推理与 Python binding，支持离线、流式和纯 CPU 模式。 | [项目 README](https://github.com/di-osc/vad-burn#readme) 和 FunASR showcase [#3106](https://github.com/modelscope/FunASR/issues/3106)。 |
4949

‎docs/deployment_matrix.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm
1515
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
1616
| ONNX/C++ runtime | High-concurrency CPU services or embedded realtime ASR | [ONNX runtime docs](../runtime/onnxruntime/readme.md) | Keep this path when latency/concurrency is already proven; add text post-processing for fixed business terms before moving to GPU LLMs. |
1717
| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |
18-
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`) or vLLM (`backend="vllm"`). The model remains published and maintained by OpenMOSS. |
18+
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model integrated with FunASR `AutoModel`; choose local HF (`backend="hf"`) or vLLM (`backend="vllm"`), or serve it independently through native SGLang Omni. SGLang Omni is not an `AutoModel` backend. The model remains published and maintained by OpenMOSS. |
1919
| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |
2020
| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |
2121
| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |

‎docs/deployment_matrix_ja.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
| Kubernetes API | Cluster service 向け internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP` から開始。公開範囲を広げる前に auth、TLS、network policy、GPU scheduling を追加します。 |
1414
| Runtime WebSocket service | Live captions、meeting、call-center stream | [Runtime service docs](../runtime/readme.md) | partial result、endpointing、long-lived audio stream が重要な場合に使います。 |
1515
| vLLM acceleration | Fun-ASR-Nano の LLM-based ASR throughput 向上 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput 向け。non-autoregressive Paraformer には適用しません。 |
16-
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF（`backend="hf"`）または vLLM（`backend="vllm"`）を選択でき、model の公開・保守主体は OpenMOSS のままです。 |
16+
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を FunASR `AutoModel` に統合済みです。local HF（`backend="hf"`）または vLLM（`backend="vllm"`）を選択でき、native SGLang Omni から独立して serve することもできます。SGLang Omni は `AutoModel` backend ではなく、model の公開・保守主体は OpenMOSS のままです。 |
1717
| MCP server | Claude/Cursor/desktop agent の speech tool | [MCP example](../examples/mcp_server/) | ASR 結果を local tool として Agent に渡したい場合に便利です。 |
1818
| Subtitle generator | 長時間 audio/video から SRT/VTT 作成 | [Subtitle example](../examples/subtitle/) | readability が重要な場合は verbose segment と speaker label を使います。 |
1919
| Batch ASR script | Archive、meeting、dataset、繰り返し offline run | [Batch example](../examples/batch_asr_improved.py) | production では queue、manifest、retry log を追加してください。 |

‎docs/deployment_matrix_ko.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
| Kubernetes API | Cluster service용 internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP`부터 시작합니다. 범위를 넓히기 전에 auth, TLS, network policy, GPU scheduling을 추가하세요. |
1414
| Runtime WebSocket service | Live captions, meeting, call-center stream | [Runtime service docs](../runtime/readme.md) | partial result, endpointing, long-lived audio stream이 중요할 때 사용합니다. |
1515
| vLLM acceleration | Fun-ASR-Nano의 LLM-based ASR throughput 향상 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput용입니다. non-autoregressive Paraformer에는 적용되지 않습니다. |
16-
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`) 또는 vLLM(`backend="vllm"`)을 선택할 수 있고, model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
16+
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model이며 FunASR `AutoModel`에 통합되어 있습니다. local HF(`backend="hf"`) 또는 vLLM(`backend="vllm"`)을 선택하거나 native SGLang Omni에서 독립적으로 serve할 수 있습니다. SGLang Omni는 `AutoModel` backend가 아니며, model의 공개 및 유지 관리는 계속 OpenMOSS가 담당합니다. |
1717
| MCP server | Claude/Cursor/desktop agent speech tool | [MCP example](../examples/mcp_server/) | ASR 결과를 local tool로 Agent에 전달할 때 유용합니다. |
1818
| Subtitle generator | 긴 audio/video에서 SRT/VTT 생성 | [Subtitle example](../examples/subtitle/) | readability가 중요하면 verbose segment와 speaker label을 사용합니다. |
1919
| Batch ASR script | Archive, meeting, dataset, 반복 offline run | [Batch example](../examples/batch_asr_improved.py) | production에서는 queue, manifest, retry log를 추가하세요. |

‎docs/deployment_matrix_zh.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
1616
| ONNX/C++ Runtime | 高并发 CPU 服务或嵌入式实时 ASR | [ONNX Runtime 文档](../runtime/onnxruntime/readme.md) | 如果延迟和并发已经验证，不要轻易替换；固定业务词优先做文本后处理。 |
1717
| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐；不适用于非自回归 Paraformer。 |
18-
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型，已接入 FunASR `AutoModel`；可选本地 HF（`backend="hf"`）或 vLLM（`backend="vllm"`）。模型仍由 OpenMOSS 发布和维护。 |
18+
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 模型，已接入 FunASR `AutoModel`；可选本地 HF（`backend="hf"`）或 vLLM（`backend="vllm"`），也可通过原生 SGLang Omni 独立服务。SGLang Omni 不是 `AutoModel` backend。模型仍由 OpenMOSS 发布和维护。 |
1919
| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |
2020
| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |
2121
| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |

‎docs/moss_transcribe_diarize.md‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -281,8 +281,18 @@ curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
281281
-F response_format=verbose_json
282282
```
283283

284-
Verify every segment has start/end timing, text, and the expected speaker
285-
field before wiring the response into subtitles, meeting notes, or analytics.
284+
Verify every segment has start/end timing and non-empty text. In SGLang Omni's
285+
current `verbose_json` contract, the speaker identifier is retained as the
286+
`[Sxx]` prefix in `segments[].text`; it is not a separate `speaker` field.
287+
Parse and validate that prefix before wiring the response into subtitles,
288+
meeting notes, or analytics.
289+
290+
The native runtime was merged in SGLang Omni
291+
[#914](https://github.com/sgl-project/sglang-omni/pull/914). Its single-H100
292+
Seed-TTS EN benchmark completed 1088/1088 clips with no request failures. WER
293+
was measured only after removing timestamp and speaker markup from
294+
single-speaker English clips, so it does not evaluate diarization or timestamp
295+
accuracy and is not a production capacity promise.
286296

287297
## Production validation
288298

‎docs/moss_transcribe_diarize_zh.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -253,7 +253,15 @@ curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
253253
-F response_format=verbose_json
254254
```
255255

256-
接入字幕、会议纪要或分析系统前，验证每个 segment 都包含起止时间、文本和预期的 speaker 字段。
256+
验证每个 segment 都有起止时间与非空文本。SGLang Omni 当前的
257+
`verbose_json` 合同把说话人编号保留为 `segments[].text` 的 `[Sxx]` 前缀，
258+
并没有单独的 `speaker` 字段。接入字幕、会议纪要或分析系统前，应解析并校验此前缀。
259+
260+
原生 runtime 已通过 SGLang Omni
261+
[#914](https://github.com/sgl-project/sglang-omni/pull/914) 合并。其单张 H100
262+
Seed-TTS EN benchmark 完成 1088/1088 条请求且无请求失败。WER 是在移除
263+
timestamp 与 speaker markup 后针对单说话人英文片段计算的，因此不评估
264+
diarization 或 timestamp 准确率，也不是生产容量承诺。
257265

258266
## 上线前验证
259267

0 commit comments

Comments
 (0)