diff --git a/docs/sdk/tutorials/audio_projects.md b/docs/sdk/tutorials/audio_projects.md
new file mode 100644
index 000000000..17c317268
--- /dev/null
+++ b/docs/sdk/tutorials/audio_projects.md
@@ -0,0 +1,453 @@
+
+
+
+# How to work with audio projects in Kili
+
+Kili audio projects are designed for **speech transcription with speaker attribution**: a labeler
+listens to a recording, draws *segments* on the waveform, types what is being said in each of them,
+and assigns each segment to a *speaker*.
+
+In this tutorial, we will go through the full life cycle of an audio project:
+
+1. Setting up an audio project
+2. Importing audio assets
+3. Understanding the audio label format
+4. Importing model pre-annotations (predictions)
+5. Exporting audio labels
+6. Cleanup
+
+Let's start by installing the SDK and instantiating the client. `Kili()` reads your API key from the
+`KILI_API_KEY` environment variable — see
+[how to create one](https://docs.kili-technology.com/docs/creating-an-api-key).
+
+
+```python
+%pip install kili
+```
+
+
+```python
+import json
+
+from kili.client import Kili
+
+kili = Kili()
+```
+
+## 1. Setting up an audio project
+
+### Designing the labeling interface
+
+An audio interface is made of two very different kinds of jobs, and it is worth understanding the
+distinction before writing any code.
+
+**Segment-level jobs.** A `TRANSCRIPTION` job that is *not* flagged as asset-level is the
+transcription job of the project. It is the job that materializes the segments drawn on the
+waveform: every segment is one annotation of that job, with a time interval, a piece of text and a
+speaker. An audio project has **exactly one** such job — it is what makes the waveform editable.
+
+**Asset-level jobs.** Any job carrying `"level": "asset"` applies to the *whole recording* rather
+than to a segment. They are rendered in a side panel on the right of the interface and behave like
+the classification and transcription jobs you already know from image or text projects. Use them for
+metadata such as the language of the call, the audio quality, or a free-text summary.
+
+Let's build an interface with one transcription job and two asset-level classification jobs.
+
+
+```python
+json_interface = {
+ "jobs": {
+ # Segment-level job: this is the job the waveform segments belong to.
+ # A TRANSCRIPTION job without "level": "asset" is the transcription job of the project.
+ "TRANSCRIPTION_JOB": {
+ "mlTask": "TRANSCRIPTION",
+ "content": {"input": "textField"},
+ "instruction": "Transcription",
+ "required": 0,
+ "isChild": False,
+ },
+ # Asset-level job: applies to the whole recording, shown in the right-hand panel.
+ "LANGUAGE_JOB": {
+ "mlTask": "CLASSIFICATION",
+ "content": {
+ "categories": {
+ "ENGLISH": {"children": [], "name": "English", "id": "category_english"},
+ "FRENCH": {"children": [], "name": "French", "id": "category_french"},
+ "OTHER": {"children": [], "name": "Other", "id": "category_other"},
+ },
+ "input": "singleDropdown",
+ },
+ "instruction": "Language of the recording",
+ "required": 1,
+ "isChild": False,
+ "level": "asset",
+ },
+ "AUDIO_CHARACTERISTICS_JOB": {
+ "mlTask": "CLASSIFICATION",
+ "content": {
+ "categories": {
+ "BACKGROUND_MUSIC": {
+ "children": [],
+ "name": "Background music",
+ "id": "category_music",
+ },
+ "BACKGROUND_NOISE": {
+ "children": [],
+ "name": "Background noise",
+ "id": "category_noise",
+ },
+ "MULTIPLE_SPEAKERS": {
+ "children": [],
+ "name": "Multiple speakers",
+ "id": "category_multi",
+ },
+ },
+ "input": "checkbox",
+ },
+ "instruction": "Audio characteristics",
+ "required": 0,
+ "isChild": False,
+ "level": "asset",
+ },
+ }
+}
+```
+
+In the project settings, Kili labels each job with the level it applies to, so you can check at a
+glance that your interface is what you intended:
+
+
+
+### Creating the project
+
+Audio projects use the `AUDIO` input type.
+
+
+```python
+project = kili.create_project(
+ title="[Kili SDK Notebook]: Audio transcription",
+ description="Speaker-attributed transcription of customer support calls",
+ input_type="AUDIO",
+ json_interface=json_interface,
+)
+
+project_id = project["id"]
+print("Project ID:", project_id)
+```
+
+ Project ID: cmsws1pir04wkvm0w0nh8fqc9
+
+
+## 2. Importing audio assets
+
+Audio assets are imported like any other asset type, with `append_many_to_dataset`.
+
+Kili accepts **`.mp3`, `.wav`, `.flac` and `.mp4`** files.
+
+### From a URL
+
+
+```python
+AUDIO_URL = "https://storage.googleapis.com/label-public-staging/demo-projects/audio/EN_Support.mp3"
+
+kili.append_many_to_dataset(
+ project_id=project_id,
+ content_array=[AUDIO_URL],
+ external_id_array=["support_call_en"],
+)
+```
+
+### From a local file
+
+To upload a file that sits on your machine, pass its path instead of a URL. Note that hosted files
+and local files cannot be mixed in a single call — use one call for each.
+
+
+```python
+import urllib.request
+
+urllib.request.urlretrieve(AUDIO_URL, "support_call.mp3")
+
+kili.append_many_to_dataset(
+ project_id=project_id,
+ content_array=["./support_call.mp3"],
+ external_id_array=["support_call_en_local"],
+)
+```
+
+## 3. Understanding the audio label format
+
+An audio `jsonResponse` has one key that no other input type has: `speakers`.
+
+```json
+{
+ "speakers": [
+ {"id": "spk_agent", "name": "Agent", "color": "#7C3AED"},
+ {"id": "spk_customer", "name": "Customer", "color": "#059669"}
+ ],
+ "TRANSCRIPTION_JOB": {
+ "annotations": [
+ {
+ "mid": "segment_000",
+ "startTime": 2.0,
+ "endTime": 2.3,
+ "speakerId": "spk_customer",
+ "text": "Hello."
+ }
+ ]
+ },
+ "LANGUAGE_JOB": {"categories": [{"name": "ENGLISH"}]},
+ "AUDIO_CHARACTERISTICS_JOB": {"categories": [{"name": "MULTIPLE_SPEAKERS"}]}
+}
+```
+
+### Speakers
+
+`speakers` is the cast of the recording. Speakers are defined **per label**, not per project: two
+assets in the same project can have completely different speakers, which is exactly what you want
+when each recording is a different conversation.
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `id` | `str` | The identifier you choose. Segments reference the speaker through it. |
+| `name` | `str` | The name displayed on the speaker tag, e.g. `Agent`. |
+| `color` | `str` | Hex color of the tag and of the segment on the waveform, e.g. `#7C3AED`. |
+
+All three fields are required. Colors are free-form hex strings; the palette the Kili interface uses
+when a labeler adds a speaker by hand is `#7C3AED`, `#2563EB`, `#059669`, `#DC2626`, `#D97706`,
+`#0891B2`, `#EC4899`, `#4F46E5`, and picking from it keeps imported labels visually consistent with
+manually created ones.
+
+### Segments
+
+The transcription job holds an `annotations` list — one entry per segment on the waveform.
+
+| Field | Type | Required | Description |
+| --- | --- | --- | --- |
+| `mid` | `str` | yes | Identifier of the segment, unique within the label. It is preserved on export, which makes it the reliable key to join a segment back to your own data. |
+| `startTime` | `float` | yes | Start of the segment, **in seconds**. |
+| `endTime` | `float` | yes | End of the segment, **in seconds**. |
+| `text` | `str` | yes | The transcription. Use `""` for a segment that still has to be transcribed. |
+| `speakerId` | `str` | no | Id of one of the entries of `speakers`. Omit it (or set it to `null`) to leave the segment unassigned — it will show up as *Unknown* in the interface. |
+
+Times are expressed in seconds as floats, with millisecond precision — Kili stores them internally as
+integer milliseconds, so `2.3456` is rounded to `2.346`.
+
+### Asset-level jobs
+
+Asset-level jobs use the classic `jsonResponse` shape you already know, keyed by job name:
+`{"categories": [{"name": "ENGLISH"}]}` for a classification, `{"text": "..."}` for a transcription.
+
+## 4. Importing model pre-annotations
+
+Speech-to-text models paired with a diarization model produce exactly the information Kili needs:
+time-aligned segments, their transcription, and which speaker uttered them. Importing them as
+**predictions** gives labelers a draft to correct instead of a blank waveform.
+
+Here we hardcode the output of such a pipeline, but in a real workflow this would come from Whisper,
+`pyannote.audio`, a cloud speech API, or your own model.
+
+
+```python
+speakers = [
+ {"id": "spk_agent", "name": "Agent", "color": "#7C3AED"},
+ {"id": "spk_customer", "name": "Customer", "color": "#059669"},
+]
+
+# (start, end, speaker, text) as produced by a transcription + diarization pipeline
+raw_segments = [
+ (2.00, 2.30, "spk_customer", "Hello."),
+ (3.90, 5.40, "spk_agent", "Hello, I'm speaking to Mariam."),
+ (6.40, 7.50, "spk_customer", "Yes, speaking."),
+ (8.20, 9.40, "spk_agent", "Hello, my name is Stephen."),
+ (9.50, 13.30, "spk_agent", "I'm calling you from the finance department."),
+ (
+ 14.20,
+ 18.10,
+ "spk_agent",
+ "You were speaking with Michael before, and your manager is Mr. Omar, correct?",
+ ),
+ (19.10, 20.10, "spk_customer", "Okay."),
+ (
+ 20.10,
+ 31.40,
+ "spk_agent",
+ "All right. I was calling you because we were trying to find a way to make a quick and easy withdrawal of your money back to your bank.",
+ ),
+ (
+ 31.70,
+ 36.30,
+ "spk_agent",
+ "I think we finally found an option, and that's why I'm calling you.",
+ ),
+ (36.80, 39.70, "spk_agent", "It will just take another five or ten minutes."),
+ (
+ 40.00,
+ 44.50,
+ "spk_agent",
+ "If you're available, I would like to guide you through the steps.",
+ ),
+ (48.30, 50.40, "spk_agent", "So are you available for me to help you with that?"),
+ (51.90, 52.30, "spk_customer", "Yes."),
+ (53.00, 54.10, "spk_agent", "Okay, wonderful."),
+]
+
+json_response = {
+ "speakers": speakers,
+ "TRANSCRIPTION_JOB": {
+ "annotations": [
+ {
+ "mid": f"segment_{index:03d}",
+ "startTime": start,
+ "endTime": end,
+ "speakerId": speaker_id,
+ "text": text,
+ }
+ for index, (start, end, speaker_id, text) in enumerate(raw_segments)
+ ]
+ },
+ # asset-level jobs, filled in the same call
+ "LANGUAGE_JOB": {"categories": [{"name": "ENGLISH"}]},
+ "AUDIO_CHARACTERISTICS_JOB": {"categories": [{"name": "MULTIPLE_SPEAKERS"}]},
+}
+```
+
+`label_type="PREDICTION"` marks the label as model output, and `model_name` records which model
+produced it, so you can later compare several models on the same assets.
+
+
+```python
+kili.append_labels(
+ project_id=project_id,
+ asset_external_id_array=["support_call_en"],
+ json_response_array=[json_response],
+ label_type="PREDICTION",
+ model_name="whisper-large-v3",
+)
+```
+
+
+
+A few things to keep in mind when building the `jsonResponse`:
+
+- **`mid` is mandatory.** Unlike bounding boxes in image projects, audio segments are not assigned an
+ identifier automatically; a segment without a `mid` is rejected.
+- **Segments do not have to be sorted.** Kili orders them by `startTime` when it returns them.
+- **Overlapping segments are allowed**, which matters when two people talk over each other.
+- **Every `speakerId` should exist in `speakers`.** A segment pointing at an unknown speaker is
+ imported, but the interface will render it as *Unknown*.
+
+To import ground-truth labels rather than predictions, use the very same `jsonResponse` with the
+default `label_type="DEFAULT"`.
+
+## 5. Exporting audio labels
+
+`kili.labels` returns the labels of a project. Beyond `jsonResponse`, audio labels expose the
+`speakers` relation, which gives you the cast of each label.
+
+
+```python
+labels = kili.labels(
+ project_id=project_id,
+ asset_external_id_in=["support_call_en"],
+ fields=[
+ "labelType",
+ "modelName",
+ "jsonResponse",
+ "speakers.id",
+ "speakers.name",
+ "speakers.color",
+ ],
+)
+
+label = labels[0]
+print(label["labelType"], "-", label["modelName"])
+print(json.dumps(label["speakers"], indent=2))
+```
+
+ PREDICTION - whisper-large-v3
+ [
+ {
+ "id": "spk_agent",
+ "name": "Agent",
+ "color": "#7C3AED"
+ },
+ {
+ "id": "spk_customer",
+ "name": "Customer",
+ "color": "#059669"
+ }
+ ]
+
+
+
+```python
+segments = label["jsonResponse"]["TRANSCRIPTION_JOB"]["annotations"]
+
+print(f"{len(segments)} segments")
+print(json.dumps(segments[:2], indent=2))
+```
+
+ 14 segments
+ [
+ {
+ "mid": "segment_000",
+ "startTime": 2,
+ "endTime": 2.3,
+ "speakerId": "cmsws1q1j04x8vm0w85204mu8",
+ "text": "Hello."
+ },
+ {
+ "mid": "segment_001",
+ "startTime": 3.9,
+ "endTime": 5.4,
+ "speakerId": "cmsws1q1j04x7vm0w5pnofrun",
+ "text": "Hello, I'm speaking to Mariam."
+ }
+ ]
+
+
+`mid`, `startTime`, `endTime` and `text` come back exactly as they were imported — `mid` in
+particular is your stable join key back to your own data.
+
+### Exporting the whole project to a file
+
+To get every asset and every label at once, use `export_labels`. The `raw` and `kili` formats keep
+the audio `jsonResponse` untouched, `speakers` relation aside; the computer-vision formats
+(`coco`, `yolo_*`, `pascal_voc`) and `geojson` do not apply to audio.
+
+By default only submitted labels are exported, so pass `label_type_in` and `export_type="normal"` if
+you also want the predictions.
+
+
+```python
+kili.export_labels(
+ project_id=project_id,
+ filename="audio_export.zip",
+ fmt="raw",
+ with_assets=False,
+ label_type_in=["DEFAULT", "PREDICTION"],
+ export_type="normal",
+)
+```
+
+## 6. Cleanup
+
+Let's remove the project we created for this tutorial.
+
+
+```python
+kili.delete_project(project_id)
+```
+
+## Summary
+
+We created an audio project, learned the difference between the segment-level transcription job and
+asset-level jobs, imported audio assets from a URL and from disk, imported speaker-attributed
+predictions, and exported them back.
+
+For more on the concepts used along the way, see:
+
+- [Importing assets](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/importing_assets_and_metadata/)
+- [Importing labels](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/importing_labels/)
+- [Exporting a project](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/export_a_kili_project/)
diff --git a/docs/tutorials.md b/docs/tutorials.md
index da6181615..e2c9a0d36 100644
--- a/docs/tutorials.md
+++ b/docs/tutorials.md
@@ -21,6 +21,7 @@ Because videos and Rich Text assets may be more complex to import, we’ve creat
- For PDF assets, see [here](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/importing_pdf_assets).
- For Geospatial multi-layer assets, see [here](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/importing_multilayer_geospatial_assets).
- For LLM Static, see [here](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/llm_static/).
+- For audio assets, see the [audio projects tutorial](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/audio_projects/), which covers importing recordings along with the rest of the audio workflow.
## Importing labels
@@ -70,6 +71,10 @@ For a more specific use case, follow [this tutorial](https://python-sdk-docs.kil
Webhooks are really similar to plugins, except they are self-hosted, and require a web service deployed at your end, callable by Kili. To learn how to use webhooks, follow [this tutorial](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/webhooks_example/).
+## Audio projects
+
+[This tutorial](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/audio_projects/) walks you through the whole life cycle of an audio transcription project: designing an interface with a segment-level transcription job and asset-level jobs, importing recordings, importing speaker-attributed pre-annotations, and exporting the result. It also explains how speakers work in the audio label format.
+
## LLM Dynamic Project
[This tutorial](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/llm_dynamic/) guides you through setting up a Kili project with an integrated LLM. You'll learn how to create and link the LLM model to the project and initiate a conversation using the Kili SDK.
diff --git a/mkdocs.yml b/mkdocs.yml
index 40df8b9d2..8c64b4f43 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -62,6 +62,7 @@ nav:
- Exporting Project Data:
- Exporting a Project: sdk/tutorials/export_a_kili_project.md
- Parsing Labels: sdk/tutorials/label_parsing.md
+ - Audio Projects: sdk/tutorials/audio_projects.md
- LLM Dynamic Projects: sdk/tutorials/llm_dynamic.md
- Setting Up Plugins:
- Developing Plugins: sdk/tutorials/plugins_development.md
diff --git a/recipes/audio_projects.ipynb b/recipes/audio_projects.ipynb
new file mode 100644
index 000000000..b72b3b856
--- /dev/null
+++ b/recipes/audio_projects.ipynb
@@ -0,0 +1,682 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\")
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# How to work with audio projects in Kili"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Kili audio projects are designed for **speech transcription with speaker attribution**: a labeler\n",
+ "listens to a recording, draws *segments* on the waveform, types what is being said in each of them,\n",
+ "and assigns each segment to a *speaker*.\n",
+ "\n",
+ "In this tutorial, we will go through the full life cycle of an audio project:\n",
+ "\n",
+ "1. Setting up an audio project\n",
+ "2. Importing audio assets\n",
+ "3. Understanding the audio label format\n",
+ "4. Importing model pre-annotations (predictions)\n",
+ "5. Exporting audio labels\n",
+ "6. Cleanup"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's start by installing the SDK and instantiating the client. `Kili()` reads your API key from the\n",
+ "`KILI_API_KEY` environment variable — see\n",
+ "[how to create one](https://docs.kili-technology.com/docs/creating-an-api-key)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%pip install kili"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import json\n",
+ "\n",
+ "from kili.client import Kili\n",
+ "\n",
+ "kili = Kili()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 1. Setting up an audio project"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Designing the labeling interface"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "An audio interface is made of two very different kinds of jobs, and it is worth understanding the\n",
+ "distinction before writing any code.\n",
+ "\n",
+ "**Segment-level jobs.** A `TRANSCRIPTION` job that is *not* flagged as asset-level is the\n",
+ "transcription job of the project. It is the job that materializes the segments drawn on the\n",
+ "waveform: every segment is one annotation of that job, with a time interval, a piece of text and a\n",
+ "speaker. An audio project has **exactly one** such job — it is what makes the waveform editable.\n",
+ "\n",
+ "**Asset-level jobs.** Any job carrying `\"level\": \"asset\"` applies to the *whole recording* rather\n",
+ "than to a segment. They are rendered in a side panel on the right of the interface and behave like\n",
+ "the classification and transcription jobs you already know from image or text projects. Use them for\n",
+ "metadata such as the language of the call, the audio quality, or a free-text summary.\n",
+ "\n",
+ "Let's build an interface with one transcription job and two asset-level classification jobs."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "json_interface = {\n",
+ " \"jobs\": {\n",
+ " # Segment-level job: this is the job the waveform segments belong to.\n",
+ " # A TRANSCRIPTION job without \"level\": \"asset\" is the transcription job of the project.\n",
+ " \"TRANSCRIPTION_JOB\": {\n",
+ " \"mlTask\": \"TRANSCRIPTION\",\n",
+ " \"content\": {\"input\": \"textField\"},\n",
+ " \"instruction\": \"Transcription\",\n",
+ " \"required\": 0,\n",
+ " \"isChild\": False,\n",
+ " },\n",
+ " # Asset-level job: applies to the whole recording, shown in the right-hand panel.\n",
+ " \"LANGUAGE_JOB\": {\n",
+ " \"mlTask\": \"CLASSIFICATION\",\n",
+ " \"content\": {\n",
+ " \"categories\": {\n",
+ " \"ENGLISH\": {\"children\": [], \"name\": \"English\", \"id\": \"category_english\"},\n",
+ " \"FRENCH\": {\"children\": [], \"name\": \"French\", \"id\": \"category_french\"},\n",
+ " \"OTHER\": {\"children\": [], \"name\": \"Other\", \"id\": \"category_other\"},\n",
+ " },\n",
+ " \"input\": \"singleDropdown\",\n",
+ " },\n",
+ " \"instruction\": \"Language of the recording\",\n",
+ " \"required\": 1,\n",
+ " \"isChild\": False,\n",
+ " \"level\": \"asset\",\n",
+ " },\n",
+ " \"AUDIO_CHARACTERISTICS_JOB\": {\n",
+ " \"mlTask\": \"CLASSIFICATION\",\n",
+ " \"content\": {\n",
+ " \"categories\": {\n",
+ " \"BACKGROUND_MUSIC\": {\n",
+ " \"children\": [],\n",
+ " \"name\": \"Background music\",\n",
+ " \"id\": \"category_music\",\n",
+ " },\n",
+ " \"BACKGROUND_NOISE\": {\n",
+ " \"children\": [],\n",
+ " \"name\": \"Background noise\",\n",
+ " \"id\": \"category_noise\",\n",
+ " },\n",
+ " \"MULTIPLE_SPEAKERS\": {\n",
+ " \"children\": [],\n",
+ " \"name\": \"Multiple speakers\",\n",
+ " \"id\": \"category_multi\",\n",
+ " },\n",
+ " },\n",
+ " \"input\": \"checkbox\",\n",
+ " },\n",
+ " \"instruction\": \"Audio characteristics\",\n",
+ " \"required\": 0,\n",
+ " \"isChild\": False,\n",
+ " \"level\": \"asset\",\n",
+ " },\n",
+ " }\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In the project settings, Kili labels each job with the level it applies to, so you can check at a\n",
+ "glance that your interface is what you intended:\n",
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Creating the project"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Audio projects use the `AUDIO` input type."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Project ID: cmsws1pir04wkvm0w0nh8fqc9\n"
+ ]
+ }
+ ],
+ "source": [
+ "project = kili.create_project(\n",
+ " title=\"[Kili SDK Notebook]: Audio transcription\",\n",
+ " description=\"Speaker-attributed transcription of customer support calls\",\n",
+ " input_type=\"AUDIO\",\n",
+ " json_interface=json_interface,\n",
+ ")\n",
+ "\n",
+ "project_id = project[\"id\"]\n",
+ "print(\"Project ID:\", project_id)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 2. Importing audio assets"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Audio assets are imported like any other asset type, with `append_many_to_dataset`.\n",
+ "\n",
+ "Kili accepts **`.mp3`, `.wav`, `.flac` and `.mp4`** files."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### From a URL"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "AUDIO_URL = \"https://storage.googleapis.com/label-public-staging/demo-projects/audio/EN_Support.mp3\"\n",
+ "\n",
+ "kili.append_many_to_dataset(\n",
+ " project_id=project_id,\n",
+ " content_array=[AUDIO_URL],\n",
+ " external_id_array=[\"support_call_en\"],\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### From a local file"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To upload a file that sits on your machine, pass its path instead of a URL. Note that hosted files\n",
+ "and local files cannot be mixed in a single call — use one call for each."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import urllib.request\n",
+ "\n",
+ "urllib.request.urlretrieve(AUDIO_URL, \"support_call.mp3\")\n",
+ "\n",
+ "kili.append_many_to_dataset(\n",
+ " project_id=project_id,\n",
+ " content_array=[\"./support_call.mp3\"],\n",
+ " external_id_array=[\"support_call_en_local\"],\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 3. Understanding the audio label format"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "An audio `jsonResponse` has one key that no other input type has: `speakers`.\n",
+ "\n",
+ "```json\n",
+ "{\n",
+ " \"speakers\": [\n",
+ " {\"id\": \"spk_agent\", \"name\": \"Agent\", \"color\": \"#7C3AED\"},\n",
+ " {\"id\": \"spk_customer\", \"name\": \"Customer\", \"color\": \"#059669\"}\n",
+ " ],\n",
+ " \"TRANSCRIPTION_JOB\": {\n",
+ " \"annotations\": [\n",
+ " {\n",
+ " \"mid\": \"segment_000\",\n",
+ " \"startTime\": 2.0,\n",
+ " \"endTime\": 2.3,\n",
+ " \"speakerId\": \"spk_customer\",\n",
+ " \"text\": \"Hello.\"\n",
+ " }\n",
+ " ]\n",
+ " },\n",
+ " \"LANGUAGE_JOB\": {\"categories\": [{\"name\": \"ENGLISH\"}]},\n",
+ " \"AUDIO_CHARACTERISTICS_JOB\": {\"categories\": [{\"name\": \"MULTIPLE_SPEAKERS\"}]}\n",
+ "}\n",
+ "```\n",
+ "\n",
+ "### Speakers\n",
+ "\n",
+ "`speakers` is the cast of the recording. Speakers are defined **per label**, not per project: two\n",
+ "assets in the same project can have completely different speakers, which is exactly what you want\n",
+ "when each recording is a different conversation.\n",
+ "\n",
+ "| Field | Type | Description |\n",
+ "| --- | --- | --- |\n",
+ "| `id` | `str` | The identifier you choose. Segments reference the speaker through it. |\n",
+ "| `name` | `str` | The name displayed on the speaker tag, e.g. `Agent`. |\n",
+ "| `color` | `str` | Hex color of the tag and of the segment on the waveform, e.g. `#7C3AED`. |\n",
+ "\n",
+ "All three fields are required. Colors are free-form hex strings; the palette the Kili interface uses\n",
+ "when a labeler adds a speaker by hand is `#7C3AED`, `#2563EB`, `#059669`, `#DC2626`, `#D97706`,\n",
+ "`#0891B2`, `#EC4899`, `#4F46E5`, and picking from it keeps imported labels visually consistent with\n",
+ "manually created ones.\n",
+ "\n",
+ "### Segments\n",
+ "\n",
+ "The transcription job holds an `annotations` list — one entry per segment on the waveform.\n",
+ "\n",
+ "| Field | Type | Required | Description |\n",
+ "| --- | --- | --- | --- |\n",
+ "| `mid` | `str` | yes | Identifier of the segment, unique within the label. It is preserved on export, which makes it the reliable key to join a segment back to your own data. |\n",
+ "| `startTime` | `float` | yes | Start of the segment, **in seconds**. |\n",
+ "| `endTime` | `float` | yes | End of the segment, **in seconds**. |\n",
+ "| `text` | `str` | yes | The transcription. Use `\"\"` for a segment that still has to be transcribed. |\n",
+ "| `speakerId` | `str` | no | Id of one of the entries of `speakers`. Omit it (or set it to `null`) to leave the segment unassigned — it will show up as *Unknown* in the interface. |\n",
+ "\n",
+ "Times are expressed in seconds as floats, with millisecond precision — Kili stores them internally as\n",
+ "integer milliseconds, so `2.3456` is rounded to `2.346`.\n",
+ "\n",
+ "### Asset-level jobs\n",
+ "\n",
+ "Asset-level jobs use the classic `jsonResponse` shape you already know, keyed by job name:\n",
+ "`{\"categories\": [{\"name\": \"ENGLISH\"}]}` for a classification, `{\"text\": \"...\"}` for a transcription."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 4. Importing model pre-annotations"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Speech-to-text models paired with a diarization model produce exactly the information Kili needs:\n",
+ "time-aligned segments, their transcription, and which speaker uttered them. Importing them as\n",
+ "**predictions** gives labelers a draft to correct instead of a blank waveform.\n",
+ "\n",
+ "Here we hardcode the output of such a pipeline, but in a real workflow this would come from Whisper,\n",
+ "`pyannote.audio`, a cloud speech API, or your own model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "speakers = [\n",
+ " {\"id\": \"spk_agent\", \"name\": \"Agent\", \"color\": \"#7C3AED\"},\n",
+ " {\"id\": \"spk_customer\", \"name\": \"Customer\", \"color\": \"#059669\"},\n",
+ "]\n",
+ "\n",
+ "# (start, end, speaker, text) as produced by a transcription + diarization pipeline\n",
+ "raw_segments = [\n",
+ " (2.00, 2.30, \"spk_customer\", \"Hello.\"),\n",
+ " (3.90, 5.40, \"spk_agent\", \"Hello, I'm speaking to Mariam.\"),\n",
+ " (6.40, 7.50, \"spk_customer\", \"Yes, speaking.\"),\n",
+ " (8.20, 9.40, \"spk_agent\", \"Hello, my name is Stephen.\"),\n",
+ " (9.50, 13.30, \"spk_agent\", \"I'm calling you from the finance department.\"),\n",
+ " (\n",
+ " 14.20,\n",
+ " 18.10,\n",
+ " \"spk_agent\",\n",
+ " \"You were speaking with Michael before, and your manager is Mr. Omar, correct?\",\n",
+ " ),\n",
+ " (19.10, 20.10, \"spk_customer\", \"Okay.\"),\n",
+ " (\n",
+ " 20.10,\n",
+ " 31.40,\n",
+ " \"spk_agent\",\n",
+ " \"All right. I was calling you because we were trying to find a way to make a quick and easy withdrawal of your money back to your bank.\",\n",
+ " ),\n",
+ " (\n",
+ " 31.70,\n",
+ " 36.30,\n",
+ " \"spk_agent\",\n",
+ " \"I think we finally found an option, and that's why I'm calling you.\",\n",
+ " ),\n",
+ " (36.80, 39.70, \"spk_agent\", \"It will just take another five or ten minutes.\"),\n",
+ " (\n",
+ " 40.00,\n",
+ " 44.50,\n",
+ " \"spk_agent\",\n",
+ " \"If you're available, I would like to guide you through the steps.\",\n",
+ " ),\n",
+ " (48.30, 50.40, \"spk_agent\", \"So are you available for me to help you with that?\"),\n",
+ " (51.90, 52.30, \"spk_customer\", \"Yes.\"),\n",
+ " (53.00, 54.10, \"spk_agent\", \"Okay, wonderful.\"),\n",
+ "]\n",
+ "\n",
+ "json_response = {\n",
+ " \"speakers\": speakers,\n",
+ " \"TRANSCRIPTION_JOB\": {\n",
+ " \"annotations\": [\n",
+ " {\n",
+ " \"mid\": f\"segment_{index:03d}\",\n",
+ " \"startTime\": start,\n",
+ " \"endTime\": end,\n",
+ " \"speakerId\": speaker_id,\n",
+ " \"text\": text,\n",
+ " }\n",
+ " for index, (start, end, speaker_id, text) in enumerate(raw_segments)\n",
+ " ]\n",
+ " },\n",
+ " # asset-level jobs, filled in the same call\n",
+ " \"LANGUAGE_JOB\": {\"categories\": [{\"name\": \"ENGLISH\"}]},\n",
+ " \"AUDIO_CHARACTERISTICS_JOB\": {\"categories\": [{\"name\": \"MULTIPLE_SPEAKERS\"}]},\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`label_type=\"PREDICTION\"` marks the label as model output, and `model_name` records which model\n",
+ "produced it, so you can later compare several models on the same assets."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "kili.append_labels(\n",
+ " project_id=project_id,\n",
+ " asset_external_id_array=[\"support_call_en\"],\n",
+ " json_response_array=[json_response],\n",
+ " label_type=\"PREDICTION\",\n",
+ " model_name=\"whisper-large-v3\",\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "A few things to keep in mind when building the `jsonResponse`:\n",
+ "\n",
+ "- **`mid` is mandatory.** Unlike bounding boxes in image projects, audio segments are not assigned an\n",
+ " identifier automatically; a segment without a `mid` is rejected.\n",
+ "- **Segments do not have to be sorted.** Kili orders them by `startTime` when it returns them.\n",
+ "- **Overlapping segments are allowed**, which matters when two people talk over each other.\n",
+ "- **Every `speakerId` should exist in `speakers`.** A segment pointing at an unknown speaker is\n",
+ " imported, but the interface will render it as *Unknown*.\n",
+ "\n",
+ "To import ground-truth labels rather than predictions, use the very same `jsonResponse` with the\n",
+ "default `label_type=\"DEFAULT\"`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 5. Exporting audio labels"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`kili.labels` returns the labels of a project. Beyond `jsonResponse`, audio labels expose the\n",
+ "`speakers` relation, which gives you the cast of each label."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "PREDICTION - whisper-large-v3\n",
+ "[\n",
+ " {\n",
+ " \"id\": \"spk_agent\",\n",
+ " \"name\": \"Agent\",\n",
+ " \"color\": \"#7C3AED\"\n",
+ " },\n",
+ " {\n",
+ " \"id\": \"spk_customer\",\n",
+ " \"name\": \"Customer\",\n",
+ " \"color\": \"#059669\"\n",
+ " }\n",
+ "]\n"
+ ]
+ }
+ ],
+ "source": [
+ "labels = kili.labels(\n",
+ " project_id=project_id,\n",
+ " asset_external_id_in=[\"support_call_en\"],\n",
+ " fields=[\n",
+ " \"labelType\",\n",
+ " \"modelName\",\n",
+ " \"jsonResponse\",\n",
+ " \"speakers.id\",\n",
+ " \"speakers.name\",\n",
+ " \"speakers.color\",\n",
+ " ],\n",
+ ")\n",
+ "\n",
+ "label = labels[0]\n",
+ "print(label[\"labelType\"], \"-\", label[\"modelName\"])\n",
+ "print(json.dumps(label[\"speakers\"], indent=2))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "14 segments\n",
+ "[\n",
+ " {\n",
+ " \"mid\": \"segment_000\",\n",
+ " \"startTime\": 2,\n",
+ " \"endTime\": 2.3,\n",
+ " \"speakerId\": \"cmsws1q1j04x8vm0w85204mu8\",\n",
+ " \"text\": \"Hello.\"\n",
+ " },\n",
+ " {\n",
+ " \"mid\": \"segment_001\",\n",
+ " \"startTime\": 3.9,\n",
+ " \"endTime\": 5.4,\n",
+ " \"speakerId\": \"cmsws1q1j04x7vm0w5pnofrun\",\n",
+ " \"text\": \"Hello, I'm speaking to Mariam.\"\n",
+ " }\n",
+ "]\n"
+ ]
+ }
+ ],
+ "source": [
+ "segments = label[\"jsonResponse\"][\"TRANSCRIPTION_JOB\"][\"annotations\"]\n",
+ "\n",
+ "print(f\"{len(segments)} segments\")\n",
+ "print(json.dumps(segments[:2], indent=2))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`mid`, `startTime`, `endTime` and `text` come back exactly as they were imported — `mid` in\n",
+ "particular is your stable join key back to your own data."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Exporting the whole project to a file"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To get every asset and every label at once, use `export_labels`. The `raw` and `kili` formats keep\n",
+ "the audio `jsonResponse` untouched, `speakers` relation aside; the computer-vision formats\n",
+ "(`coco`, `yolo_*`, `pascal_voc`) and `geojson` do not apply to audio.\n",
+ "\n",
+ "By default only submitted labels are exported, so pass `label_type_in` and `export_type=\"normal\"` if\n",
+ "you also want the predictions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "kili.export_labels(\n",
+ " project_id=project_id,\n",
+ " filename=\"audio_export.zip\",\n",
+ " fmt=\"raw\",\n",
+ " with_assets=False,\n",
+ " label_type_in=[\"DEFAULT\", \"PREDICTION\"],\n",
+ " export_type=\"normal\",\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 6. Cleanup"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Let's remove the project we created for this tutorial."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "kili.delete_project(project_id)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Summary"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We created an audio project, learned the difference between the segment-level transcription job and\n",
+ "asset-level jobs, imported audio assets from a URL and from disk, imported speaker-attributed\n",
+ "predictions, and exported them back.\n",
+ "\n",
+ "For more on the concepts used along the way, see:\n",
+ "\n",
+ "- [Importing assets](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/importing_assets_and_metadata/)\n",
+ "- [Importing labels](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/importing_labels/)\n",
+ "- [Exporting a project](https://python-sdk-docs.kili-technology.com/latest/sdk/tutorials/export_a_kili_project/)"
+ ]
+ }
+ ],
+ "metadata": {
+ "language_info": {
+ "name": "python"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/recipes/img/audio_jobs_settings.png b/recipes/img/audio_jobs_settings.png
new file mode 100644
index 000000000..dfdee0c33
Binary files /dev/null and b/recipes/img/audio_jobs_settings.png differ
diff --git a/recipes/img/audio_labeling_interface.png b/recipes/img/audio_labeling_interface.png
new file mode 100644
index 000000000..0e313564e
Binary files /dev/null and b/recipes/img/audio_labeling_interface.png differ