Not able to get diarization + ASR working

[pmarini-nc]

My goal is to have a meeting transcribed with diarization. I have the meeting recording in a file file.mp3.

The following call:

curl -X 'POST'   'https://<my-server>/v1/audio/diarization'   -H 'accept: application/json'   -H 'Content-Type: multipart/form-data'   -F 'model=vibevoice-crispasr'   -F 'file=@file.mp3;type=audio/mpeg'   -F 'include_text=true' -H "Authorization: Bearer <MYTOKEN>"

returns:

{"error":{"code":500,"message":"rpc error: code = Unknown desc = unimplemented","type":""}}

vibevoice-crispasr is running:

Configuration file of the model:

backend: cpu-crispasr
known_usecases:
    - transcript
name: vibevoice-crispasr
parameters:
    model: vibevoice-asr-q4_k.gguf

I'm on LocalAI 4.5.0. I've went through https://localai.io/features/audio-diarization/index.html, there are some aspects I don't fully understand and it would be great to have an end-to-end tutorial to have transcript+diarization tutorial.

Thanks!

[cananoo]

The error is caused by the selected model/backend, not by the multipart request.

vibevoice-crispasr is configured as an ASR model using the CrispASR backend. Its gallery configuration declares transcript as the use case, but it is not the diarization-capable VibeVoice backend. Adding include_text=true to that model cannot enable speaker diarization, so the diarization endpoint returns unimplemented.

For transcript plus speaker-labelled segments, use the native VibeVoice entry instead:

local-ai run vibevoice-cpp-asr

Then call the endpoint with that model name:

curl -X POST http://localhost:8080/v1/audio/diarization \\
  -H "Content-Type: multipart/form-data" \\
  -F "file=@file.mp3;type=audio/mpeg" \\
  -F "model=vibevoice-cpp-asr" \\
  -F "include_text=true" \\
  -F "response_format=verbose_json"

vibevoice-cpp-asr is the gallery configuration backed by vibevoice.cpp and is the documented quick-start model for this endpoint. It returns speaker-labelled segments and can attach the transcript to each segment. If you only need speaker turns and want to keep ASR separate, sherpa-onnx is the other documented option.

The relevant distinction is:

• vibevoice-crispasr -> CrispASR -> transcription only in the supplied gallery config
• vibevoice-cpp-asr -> vibevoice.cpp -> transcription plus diarization

References:

• https://github.com/mudler/LocalAI/blob/master/docs/content/features/audio-diarization.md
• https://github.com/mudler/LocalAI/blob/master/gallery/index.yaml
• https://github.com/CrispStrobe/CrispASR