Get Started#

This page is the entry point for running the Audio Analyzer microservice. Pick one of the two deployment paths and follow the linked guide.

Application Overview#

The Audio Analyzer microservice provides automatic speech recognition (ASR) and optional speaker diarization. It captures and transcribes audio, optionally identifying different speakers in the same audio chunk. Optional sentiment analysis per chunk complements the transcription output.

If you enable speaker diarization in config.yaml (models.asr.diarization: true), you must provide a Hugging Face access token (HF_TOKEN) and accept the Pyannote speaker-diarization model license on Hugging Face if you want diarization to initialize successfully. If diarization setup is incomplete, the service continues running and logs a warning while disabling diarization for that session.

Before You Begin#

Choose Deployment Path#

The container image exposes the API on host port 8010 and mounts shared folders for models, chunks, storage, and the Hugging Face cache. Fresh clones include placeholder directories for these mount roots. If you delete them and then start Compose, Docker may recreate the missing host paths as root before the container starts.

See Run with Docker Compose for the full step-by-step guide.

Quick start:

docker compose up -d --build
curl --noproxy '*' http://127.0.0.1:8010/health

If you hit permission errors on models/, chunks/, storage/, or .cache/huggingface/, see Troubleshooting.

Device visibility note: The Docker Compose flow is the verified path for GPU and NPU acceleration. The container image includes the OpenVINO GPU and NPU runtime libraries and exposes /dev/dri by default, so openvino.Core().available_devices inside the container reports CPU, GPU, and NPU (when the host drivers are present).

When running directly from the host .venv without the full Intel GPU/NPU runtime stack installed on the host, OpenVINO may report only CPU. In that case, starting the service with device: GPU or device: NPU fails fast with:

RuntimeError: Configured OpenVINO ASR device 'GPU' is not visible in this runtime.

This is a host runtime environment limitation — no application or configuration change is required. Use Docker Compose for the accelerator-enabled setup.

Run the service directly with Python. This path is useful for development or when you do not want to use Docker.

See Run on the Host for the full step-by-step guide.

Quick start:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python main.py

Verify#

Once the service is running:

curl --noproxy '*' http://127.0.0.1:8010/health

Expected response:

{"status": "ok"}

Verify batch transcription#

Transcribe a file and get a single JSON response:

curl --noproxy '*' -X POST http://127.0.0.1:8010/v1/audio/transcriptions \
  -F file=@tests/philosophy_10_russell_128kb.mp3

Expected result: a JSON body containing a "text" field, plus an X-Session-ID response header.

Verify OpenAI-compatible streaming (SSE)#

Add stream=true to receive incremental Server-Sent Events instead of waiting for the whole file. Use -N so curl does not buffer the stream:

curl --noproxy '*' -N -X POST http://127.0.0.1:8010/v1/audio/transcriptions \
  -F file=@tests/philosophy_10_russell_128kb.mp3 \
  -F stream=true

Expected result: a sequence of transcript.text.delta frames as each chunk is transcribed, one final transcript.text.done frame with the full text, then the [DONE] sentinel:

data: {"type": "transcript.text.delta", "delta": "Chapter X of Philosophy by Bertrand Russell..."}

data: {"type": "transcript.text.delta", "delta": "The electron, which has been moving in one orbit..."}

data: {"type": "transcript.text.done", "text": "Chapter X of Philosophy by Bertrand Russell...", "language": "en", "duration": 612.4}

data: [DONE]

Because this matches OpenAI’s documented event shape, official OpenAI SDKs work against this endpoint — point the client’s base_url at this service:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8010/v1", api_key="not-used")

with open("tests/philosophy_10_russell_128kb.mp3", "rb") as audio:
    stream = client.audio.transcriptions.create(
        model="whisper-1", file=audio, response_format="json", stream=True,
    )
    for event in stream:
        print(event)

stream=true requires response_format to be json or verbose_json; srt, vtt, and text return HTTP 400.

Verify continuous audio streaming (WebSocket)#

WS /v1/realtime?intent=transcription accepts a live, continuous audio feed. Clients push PCM16 frames and receive transcripts as each utterance completes — server-side voice activity detection (VAD) decides where utterances end.

Install a WebSocket client, decode some audio to raw PCM16, and stream it:

pip install websockets

# Decode 18s of audio to raw PCM16, mono, 16 kHz
ffmpeg -v error -i tests/philosophy_10_russell_128kb.mp3 -t 18 \
  -f s16le -ar 16000 -ac 1 /tmp/audio.raw

import asyncio, base64, json, websockets

SAMPLE_RATE = 16000

async def main():
    pcm = open("/tmp/audio.raw", "rb").read()
    silence = b"\x00\x00" * int(SAMPLE_RATE * 1.2)   # triggers VAD end-of-speech

    url = "ws://127.0.0.1:8010/v1/realtime?intent=transcription"
    async with websockets.connect(url, max_size=None) as ws:
        print(json.loads(await ws.recv())["type"])   # transcription_session.created

        async def send():
            frame = int(SAMPLE_RATE * 0.1) * 2       # 100 ms frames
            for i in range(0, len(pcm), frame):
                await ws.send(json.dumps({
                    "type": "input_audio_buffer.append",
                    "audio": base64.b64encode(pcm[i:i + frame]).decode(),
                }))
                await asyncio.sleep(0.005)
            await ws.send(json.dumps({
                "type": "input_audio_buffer.append",
                "audio": base64.b64encode(silence).decode(),
            }))

        asyncio.create_task(send())
        while True:
            msg = json.loads(await ws.recv())
            print("EVENT:", msg["type"])
            if msg["type"] == "conversation.item.input_audio_transcription.completed":
                print("TRANSCRIPT:", msg["transcript"])
                break

asyncio.run(main())

Expected event sequence:

transcription_session.created
input_audio_buffer.speech_started
input_audio_buffer.speech_stopped
input_audio_buffer.committed
conversation.item.input_audio_transcription.delta
conversation.item.input_audio_transcription.completed
TRANSCRIPT: Chapter X of Philosophy by Bertrand Russell...

If you never see speech_started, the audio is quieter than the VAD threshold — lower it with a session.update message:

{"type": "session.update",
 "session": {"turn_detection": {"threshold": 0.005, "silence_duration_ms": 700}}}

To disable VAD entirely and control utterance boundaries yourself, send "turn_detection": null and then {"type": "input_audio_buffer.commit"} whenever you want a transcript.

See the API Reference for the full event list and configuration options.

Verify VSS-compatible endpoints#

Video Search & Summarization (VSS) calls this service under an /api/v1 prefix:

curl --noproxy '*' http://127.0.0.1:8010/api/v1/models

Expected: a JSON body with models and default_model. The same routes are also served unprefixed (/models, /transcriptions) for local use.

Run the automated tests#

pip install pytest httpx
pytest tests/test_streaming_endpoints.py tests/test_vss_endpoints.py -v

These cover the SSE event sequence, the realtime WebSocket handshake, VAD behavior, and the VSS contract, and require no model weights or GPU.

Next Steps#