Configuration#
kiosk-core and kiosk-ui are configured through environment variables
(see Environment Variables).
The three model-hosting services (audio-analyzer, text-to-speech,
rag-service) are configured through YAML files that the kiosk pins
and mounts into the containers. The most common changes are the
model and the inference device.
Model Selection#
Each model-hosting service reads the model identifier from the same pinned configuration file used for device selection:
Service |
File |
Model fields |
|---|---|---|
|
|
|
|
|
|
|
|
Use Hugging Face IDs where the field name is hf_id. Models are
downloaded and exported on first start into the per-service models/
directory; subsequent starts reuse the cache.
Supported and validated models#
The kiosk ships with the following defaults. These are the models the stack has been validated with — they are the recommended starting point. The Devices column lists the supported inference devices for each:
Service |
Field |
Default (validated) |
Other examples |
Devices |
|---|---|---|---|---|
|
|
|
|
|
|
|
|
other SpeechBrain emotion-recognition models |
|
|
|
|
|
|
|
|
|
other OpenVINO™-exportable instruct LLMs |
|
|
|
|
|
|
|
|
|
|
|
Important
Changing models is at your own discretion. The defaults above are the only combinations validated with this stack. Configuring models, variants, devices, or precisions other than the defaults may negatively affect the functionality, accuracy, latency, or stability of the application. You are responsible for ensuring the configuration you choose is correct and works for your use case — make changes only if you understand the implications.
In particular:
Some models do not function properly at aggressive quantization. If a model produces garbled, empty, or low-quality output at
int4, switch that model’sweight_format/dtypetoint8orfp16.A model must be exportable to the OpenVINO™ Toolkit (OpenVINO™) Intermediate Representation (IR) format for the OpenVINO™ backend; not every Hugging Face model is supported.
Larger models increase first-run download/export time, memory use, and per-request latency, and may not fit on the selected device.
After any change, restart the affected service and verify it loads and responds correctly before relying on it.
Inference Device#
Each model-hosting service reads its device from a pinned configuration file:
Service |
File |
Fields |
|---|---|---|
|
|
|
|
|
|
|
|
The supported devices for each model are listed in the Supported / validated models table above.
Use uppercase device names (CPU, GPU, and — for audio-analyzer ASR and
queue-service only — NPU). rag-service expects them as quoted strings;
for audio-analyzer and text-to-speech — unquoted.
Important
text-to-speech does not support NPU. models.tts.device only
accepts CPU/GPU (see configs/text-to-speech/config.yaml); there is no
NPU device mapping for this service in docker-compose.yml. Do not set
models.tts.device: NPU — it is not a supported configuration.
After editing, restart the affected service and confirm OpenVINO™ picked the device:
docker compose up -d --build --force-recreate <service-name>
docker compose logs <service-name> | grep -i -E "device|compiling|GPU|CPU"
OpenVINO™ prints a Compiling model on <DEVICE> line on first load.
Note
GPU execution is delegated to the OpenVINO™ backend used by each service. Whether a given model actually runs on GPU and how it performs depends on the OpenVINO™ version and operator coverage for that model.
Audio Analyzer ASR Provider/Device (config.yaml)#
Configure ASR in configs/audio-analyzer/config.yaml (single source of truth):
models.asr.providermodels.asr.device
make check-env validates this before startup and rejects unavailable
hardware early. For NPU, ACCEL_MOUNT_PATH must be set manually (in
.env or the shell) to the host NPU device node (/dev/accel/accel0)
before running make up or docker compose up — neither make up nor
make check-env auto-detects it. The checked-in docker-compose.yml
defaults ACCEL_MOUNT_PATH to /dev/null so CPU/GPU-only hosts stay
unaffected.
ASR on NPU: whisper-tiny/whisper-base only#
Important
NPU works for ASR only with models.asr.name: whisper-tiny or
whisper-base. whisper-small/medium/large fail to compile on NPU
with Check '!self_attn_nodes.empty()' failed.
Why: OpenVINO™ NPU only supports static-shape models (see
OpenVINO™ NPU docs).
Whisper’s IR has dynamic shapes, so OpenVINO™ GenAI’s NPUW plugin
pattern-matches attention blocks to make it static — a heuristic that
succeeds for tiny/base and fails for larger models. This is not fixable via
configuration; retest if you upgrade OpenVINO™/GenAI or the audio-analyzer image.
models:
asr:
provider: openvino
device: NPU
name: whisper-base # or whisper-tiny — whisper-small+ fails to compile
weight_format: null
Other ASR configurations#
# OpenAI + CPU
models: { asr: { provider: openai, device: CPU } }
# OpenVINO + GPU (any model size)
models: { asr: { provider: openvino, device: GPU } }
# OpenVINO + CPU (any model size)
models: { asr: { provider: openvino, device: CPU } }
openai supports CPU only — openai + GPU and openai + NPU are not supported.
ASR support matrix#
Provider |
CPU |
GPU |
NPU |
|---|---|---|---|
|
Yes |
No |
No |
|
Yes |
No |
No |
|
Yes |
Yes (Intel GPU required) |
|
If GPU is configured and unavailable on the host, make check-env fails before startup — no silent fallback.
Audio Analyzer Diarization Device (config.yaml)#
Important
Diarization requires three gated Hugging Face models. Before enabling it,
set HF_TOKEN in .env and accept the license on all three pages with
the same account that owns the token:
pyannote/speaker-diarization-community-1,
pyannote/speaker-diarization-3.1,
and pyannote/segmentation-3.0.
The diarization pipeline pulls segmentation-3.0 and
speaker-diarization-community-1 in addition to the configured
speaker-diarization-3.1, so accepting only the configured model still
fails with a 401. (pyannote/wespeaker-voxceleb-resnet34-LM is also
downloaded but is not gated — no license acceptance needed.) Set
KIOSK_CORE_DIARIZATION_ENABLED=false (the default) to run without diarization.
Diarization (models.diarization.device) is a separate component from ASR
(see ASR Support Matrix above) with its own,
more limited device support. ASR’s CPU/GPU/NPU support
does not apply to diarization.
Important
In the currently released Kiosk image, diarization only supports CPU.
The diarizer (pyannote/speaker-diarization-3.1, a PyTorch/SpeechBrain
model) is loaded with torch.device(<configured value>). PyTorch has no
"gpu" device string (Intel GPU support in PyTorch requires "xpu", which
this component does not use) and no "npu" device string at all. Setting
device: GPU or device: NPU is accepted by the configuration schema but
fails at diarizer-load time with an error like:
Expected one of cpu, cuda, ipu, xpu, ... device type at start of device string: gpu
This is non-fatal: the failure is caught, logged as a warning, and diarization is disabled for that session — the container stays healthy and ASR keeps working, but speaker labels are not produced.
Use device: CPU for diarization. Do not configure GPU or NPU for
models.diarization.device — they do not work in this image and will
silently disable diarization rather than accelerate it.
An OpenVINO™-backed diarization path that genuinely supports GPU/NPU
exists in a newer upstream edge-ai-libraries audio-analyzer checkout,
but is not part of the currently released Kiosk image covered by this
document. Do not configure GPU/NPU for diarization based on that
upstream code until a Kiosk image that includes it is released.
Queue Service Device (QUEUE_DEVICE)#
queue-service runs the YOLO26 person detector through a Deep Learning Streamer
(gvainference) pipeline. The inference device is controlled by
model.device in queue-service/conf/queue-config.yaml, and can be
overridden without editing that file via QUEUE_DEVICE in .env
(mapped by docker-compose.yml to QUEUE_SERVICE__MODEL__DEVICE, read by
queue-service/src/config_loader.py).
Default:
QUEUE_DEVICE=CPU— always available, no extra device mapping needed.QUEUE_DEVICE=GPU/QUEUE_DEVICE=NPU— supported, using the same/dev/driandACCEL_MOUNT_PATH-driven/dev/accelmapping described under Audio Analyzer ASR Provider/Device above. For NPU, setACCEL_MOUNT_PATHyourself to the host NPU device node before startingqueue-service— it is not auto-detected.
Verify the configured device actually reached the pipeline:
docker logs queue-service 2>&1 | grep "gvainference model"
# Expected: ... gvainference model=... device=NPU ... (or CPU/GPU, matching QUEUE_DEVICE)
Note
Editing queue-config.yaml’s model.device directly also works and takes
precedence in the same way as any other YAML default — QUEUE_DEVICE only
needs to be set when you want to override it without touching the file.
Identity Service Device (IDENTITY_DEVICE)#
identity-service performs face detection/re-identification
(face-detection-retail-0005, face-reidentification-retail-0095) and
voice-print embedding (ecapa-tdnn-voice) — all three are OpenVINO™ IR
models, loaded via openvino.Core().compile_model(model, device), and
IDENTITY_DEVICE is correctly wired end-to-end from .env through
docker-compose.yml to the OpenVINO™ compile call. The device-selection
code itself has no bug and no model-format limitation.
Note
Face detection/re-identification support CPU, GPU, and NPU.
The identity-service container has NPU device passthrough via the same
ACCEL_MOUNT_PATH//dev/accel mechanism used by audio-analyzer and
queue-service. Set IDENTITY_DEVICE=NPU in .env together with
ACCEL_MOUNT_PATH pointing to the host NPU device node — this is not
auto-detected and must be set manually — to run face
detection/re-identification on the NPU.
Voice-print embedding (ecapa-tdnn-voice) does not support NPU.
Its OpenVINO™ IR contains an internal STFT reshape with an unbounded
dynamic dimension (aten::view/Reshape), which the NPU compiler rejects
at compile time (Got negative shape dim bound). With
IDENTITY_DEVICE=NPU, the service starts with face engine enabled and
voice engine disabled (inference_ready=false, since voice verification
requires both). Use IDENTITY_DEVICE=CPU or IDENTITY_DEVICE=GPU if
voice authentication is required.
The face/voice model files are not downloaded by default — run
./setup_models.sh --identity first. Without them, the face/voice engines
stay disabled (inference_ready=false) regardless of the configured device.
Stack Inference Device (TARGET_DEVICE)#
TARGET_DEVICE is the single switch for inference hardware across the stack.
It selects the device for the ovms-llm container (the LLM served by
OpenVINO™ Model Server for the ordering agent) and, unless individually
overridden, for audio-analyzer ASR, rag-service embedding/reranker and
identity-service.
Per-service variables (ASR_DEVICE, RAG_EMBEDDING_DEVICE,
RAG_RERANKER_DEVICE, IDENTITY_DEVICE) are left commented out in
.env.example so they inherit TARGET_DEVICE. Setting any of them pins that
service and takes precedence over TARGET_DEVICE. TTS_DEVICE,
DIARIZATION_DEVICE and QUEUE_DEVICE remain CPU by design.
Note
Running without a usable GPU: run make setup-models TARGET_DEVICE=CPU.
That switches the whole stack to CPU in one step: it sets TARGET_DEVICE
and comments out any per-service *_DEVICE variable still pinned to GPU
in your existing .env. The second part matters on an install created from
an older .env.example, which shipped those variables pinned to GPU:
a per-service pin wins over TARGET_DEVICE, so setting TARGET_DEVICE=CPU
by hand is not sufficient on its own, and every pinned service
crash-loops at startup with [GPU] Context was not initialized for 0 device.
A deliberate non-GPU override (e.g., ASR_DEVICE=NPU) is left untouched.
make check-env lists the effective device per service; on a host with no
GPU render node it fails on any leftover GPU pin rather than letting
make up proceed into a crash-loop.
Supported:
TARGET_DEVICE=CPU,TARGET_DEVICE=GPU.Not supported:
TARGET_DEVICE=NPU— see below.
Important
TARGET_DEVICE=NPU is not supported for ovms-llm.
Device passthrough and model compilation both work — the ovms-llm
container has NPU passthrough via the same ACCEL_MOUNT_PATH//dev/accel
mechanism used by audio-analyzer and queue-service, OVMS logs
Available devices for Open VINO: CPU, GPU, NPU, and Qwen3-4B-int8-ov
compiles successfully. Generation is nevertheless unusable:
INT8 is numerically correct but takes 191–801 s per call, and a single voice turn issues several. That is roughly 100× too slow.
INT4 (
OpenVINO/Qwen3-4B-int4-ov) is fast (~8 s) but numerically broken on NPU — it emits repeated tokens instead of text and never forms a tool call.The NPU-optimized channel-wise tier (
int4-cw) is not published for Qwen3-4B, only Qwen3-8B.
Out of the box you will instead hit HTTP 400 — Input length exceeds the maximum allowed length, because the NPU plugin caps prompts at 1024 tokens.
Full measurements and analysis:
TARGET_DEVICE=NPU troubleshooting
(ITEP-96031). setup_models.sh --device NPU refuses to run for this reason;
use --device NPU --skip-ovms to target the NPU with the other services.
Use TARGET_DEVICE=GPU (recommended) or TARGET_DEVICE=CPU.
This restriction applies only to the OVMS-served LLM — queue-service,
identity-service, and whisper-tiny/whisper-base ASR still use the NPU.
RAG Service Embedding/Reranker Device (RAG_EMBEDDING_DEVICE, RAG_RERANKER_DEVICE)#
rag-service’s embedding (BAAI/bge-large-en-v1.5) and reranker
(BAAI/bge-reranker-base) components are OpenVINO™ IR models exported
in-process by optimum-intel (rag-service/utils/ensure_model.py) and
loaded via OVModelForFeatureExtraction/equivalent
(rag-service/components/embedding_component.py,
reranker_component.py). Their device defaults to TARGET_DEVICE and can be
pinned independently via RAG_EMBEDDING_DEVICE / RAG_RERANKER_DEVICE in
.env (mapped by docker-compose.yml to
SMART_KIOSK_RAG__MODELS__EMBEDDING__DEVICE /
SMART_KIOSK_RAG__RETRIEVAL__RERANKER__DEVICE). An explicit value wins over
TARGET_DEVICE, so leave them commented out unless you need the split.
Currently supported:
RAG_EMBEDDING_DEVICE/RAG_RERANKER_DEVICE=CPUorGPU. Default:GPU.Currently unsupported:
NPU.
Important
NPU is not supported for the embedding/reranker models.
optimum-intel’s default export produces OpenVINO™ IR with dynamic
(unbounded) sequence-length and batch shapes — required because queries
and knowledge-base documents vary in length and the reranker batches
multiple candidates per call (rag-service/config.yaml’s
models.embedding.batch_size). The NPU compiler rejects this IR
(Missing upper bound for one or more nodes); setting either variable to
NPU will crash rag-service at startup.
Forcing static/bounded shapes to work around this is not recommended:
it would require padding every input to a fixed max length (wasting
compute on short queries) and serializing what is currently a batched
reranker call into one NPU invocation per candidate — likely slower
overall than GPU/CPU, for a component that is not the latency
bottleneck (the LLM is). CPU is normally fast enough for
embedding/reranking; GPU is the default to match prior behavior.
Device-Combination Benchmarking (PROFILE)#
To compare inference devices (ASR on CPU/GPU/NPU against TTS on CPU/GPU/NPU, with
the LLM pinned to GPU) you do not need to edit any mounted config.yaml.
The variables below override the audio-analyzer and text-to-speech configurations
from the environment. Every default reproduces the checked-in YAML exactly,
so an ordinary make up is unaffected.
Variable |
Default (= shipped configuration) |
Purpose |
|---|---|---|
|
|
ASR checkpoint |
|
|
ASR final-commit device |
|
|
Streaming/partial ASR pool device |
|
|
Diarization device (CPU only) |
|
|
TTS model |
|
|
TTS engine/runtime |
|
|
TTS device |
|
|
Voice (must match the model) |
|
|
TTS precision (GPU requires |
|
|
Model variant |
|
|
Results bind mount (set by |
|
|
Metrics bind mount (set by |
The supported workflow is a profile — an environment file under
configs/benchmark-profiles/ holding one device combination:
make list-profiles
make validate-profile PROFILE=asr-npu_tts-cpu # pre-flight only, no bring-up
make benchmark PROFILE=asr-npu_tts-cpu # -> results/asr-npu_tts-cpu/
make benchmark-matrix # every profile + comparison
Each profile writes to its own results/<profile>/ and metrics/<profile>/
directory. Never point two profiles at one directory: the metrics consolidator
matches extractors by filename substring and would silently average unrelated
runs into a single meaningless row.
Important
make validate-profile is mandatory and runs automatically before make benchmark.
Besides rejecting impossible combinations, it catches the
dangerous failure mode: docker-compose.yml mounts
${ACCEL_MOUNT_PATH:-/dev/null}, so with the portable default an “NPU”
profile silently runs on CPU, succeeds, and files plausible numbers under
an NPU label. The validator errors if ACCEL_MOUNT_PATH is unset, /dev/null
or missing, and verifies the host actually exposes the requested GPU/NPU.
Because the production defaults cannot span all three devices
(distil-small.en corrupts on NPU, whisper-small, or bigger, fails to compile on NPU,
and Kokoro is CPU-only), the matrix pins whisper-base and
microsoft/speecht5_tts on the openvino runtime so that device is the only
variable. baseline-production measures the shipped configuration and should
be compared on absolute latency only — it is not the asr-gpu_tts-cpu cell.
See configs/benchmark-profiles/README.md for the full compatibility matrix.
Environment Variables#
kiosk-core has no configuration file. All settings are controlled through environment variables.
kiosk-core API (main:app)#
Variable |
Default |
Description |
|---|---|---|
|
|
audio-analyzer transcription endpoint |
|
|
RAG query endpoint |
|
|
TTS speech synthesis endpoint |
|
|
Model name sent to the TTS service |
|
(unset) |
Voice name sent to the TTS service |
|
|
Language sent to the TTS service |
|
(unset) |
Optional style instructions for TTS |
|
|
Default audio sample rate in Hz |
|
|
Length of each audio chunk sent to audio-analyzer |
|
|
Silence duration after speech that ends a session |
|
|
Hard cap on session duration |
|
|
RMS threshold below which audio is treated as silence |
|
|
PortAudio capture block size |
|
|
Audio buffered before speech starts |
|
|
HTTP client timeout for downstream calls |
Kiosk UI runtime mode {#kiosk_ui_mode}#
The React kiosk UI (kiosk-ui/) ships as a single image that can serve
either of two screens, selected at container start, and this requires no rebuild.
KIOSK_UI_MODE is the only variable the UI container reads; the browser
reaches kiosk-core and the other services through the nginx reverse
proxy baked into the image:
Variable |
Default |
Description |
|---|---|---|
|
|
|
The value is written to /usr/share/nginx/html/config.js by
docker-entrypoint.sh (installed as an nginx docker-entrypoint.d
script) and read by the SPA before the React bundle loads. In
docker-compose.yml the two screens are separate containers
(kiosk-ui and kiosk-ui-customer) built from the same image/context,
published on different host ports (7860 and 7861, respectively) so
they can be shown on two separate monitors during a demo.
Compose Defaults#
When running with the top-level docker-compose.yml, the defaults are wired to the internal Compose network:
KIOSK_CORE_ANALYZER_URL=http://audio-analyzer:8010/v1/audio/transcriptionsKIOSK_CORE_RAG_URL=http://rag-service:8020/api/v1/queryKIOSK_CORE_TTS_URL=http://text-to-speech:8011/v1/audio/speech
Most deployments should leave these values unchanged. Override them only when kiosk-core or kiosk-ui must call services outside the local Compose stack.
Session Parameters#
Session parameters (chunk duration, silence threshold, etc.) can also be provided per-request in the POST body for /api/v1/sessions/start and /api/v1/sessions/start-file. Per-request values take precedence over the environment variable defaults.
NPU Deployment Workflow#
Step-by-step workflow to run this stack with Intel NPU acceleration where supported.
NPU support by component:
Component |
NPU |
|---|---|
Queue Service ( |
✅ Yes |
Identity Service face detection/re-identification ( |
✅ Yes |
Identity Service voice/ECAPA-TDNN ( |
❌ No — dynamic shape rejected by NPU compiler |
Audio Analyzer ASR ( |
⚠️ |
Audio Analyzer Diarization ( |
❌ No — CPU only |
OVMS-LLM ( |
❌ No — compiles, but INT8 is ~100× too slow and INT4 is numerically broken — see |
RAG Service embedding/reranker ( |
❌ No — dynamic shape rejected by NPU compiler |
Text-to-Speech ( |
❌ No |
Do not set NPU for a ❌ component — it will fail to start or silently disable the feature.
The steps below configure NPU for queue-service; adapt the device variable for other ✅/⚠️ components.
1 — System requirements#
Requirement |
Details |
|---|---|
Hardware |
Intel® Core™ Ultra Series 1 or later, with integrated NPU |
Host driver |
Intel NPU driver ( |
User-space runtime |
|
Host device |
|
OpenVINO™ |
Container image already bundles the correct runtime |
Verify the NPU device node is present before proceeding:
ls /dev/accel/
# Expected: accel0 accelmon0
2 — Install the Intel NPU driver (if not already installed)#
Refer to the Intel NPU driver repository. Installation varies by distribution. After installation:
# Verify kernel driver is loaded
lsmod | grep intel_vpu
# Verify device node exists
ls -la /dev/accel/accel0
3 — Set NPU device for queue-service (and optionally audio-analyzer ASR)#
# .env
QUEUE_DEVICE=NPU
Optionally, also enable NPU for ASR (whisper-tiny/whisper-base only) in configs/audio-analyzer/config.yaml:
models:
asr:
provider: openvino
device: NPU
name: whisper-base # whisper-tiny also works; whisper-small+ fails to compile
weight_format: null
4 — Set ACCEL_MOUNT_PATH and start the stack#
ACCEL_MOUNT_PATH is not auto-detected by make up or
make check-env — export it yourself before starting the stack:
export ACCEL_MOUNT_PATH=/dev/accel/accel0
cd smart-kiosk-assistant
make check-env
make up
Alternatively, set ACCEL_MOUNT_PATH=/dev/accel/accel0 directly in
.env so it’s picked up automatically on every make up /
docker compose up without exporting it each time.
5 — Verify NPU is active#
docker ps --filter "name=queue-service" --format "{{.Names}}\t{{.Status}}"
docker exec queue-service python3 -c "import openvino as ov; print(ov.Core().available_devices)" # expect: includes NPU
docker logs queue-service 2>&1 | grep -i "device=NPU\|npu"
If ASR is on NPU (whisper-tiny/whisper-base only):
docker logs audio-analyzer 2>&1 | grep -i "Loading Model" # expect: device=NPU
curl -s -X POST http://localhost:8010/v1/audio/transcriptions -F "file=@your_sample.wav"
If identity-service face detection and re-identification run on NPU:
IDENTITY_DEVICE=NPU make up IDENTITY=true
docker logs identity-service 2>&1 | grep -i "face\|voice\|inference_ready"
6 — Troubleshooting#
Symptom |
Cause |
Fix |
|---|---|---|
Container unhealthy, |
NPU driver not loaded or |
Verify host driver and set |
|
NPU compiler not in image |
Rebuild the affected image with NPU user-space packages |
Slow first inference (20–60 s) |
NPU compiler cache is empty (cold start) |
Normal on first run; subsequent requests will be fast |
|
Model is |
Use |
Non-NPU containers unhealthy after NPU configuration change |
NPU-unrelated services picking up the wrong environment variables |
Only modify the specific component’s configuration (e.g., |
Note
Cold start: The OpenVINO™ NPU compiler caches compiled kernels inside the container under /tmp/ov_cache/. The first inference after a container restart takes significantly longer (20–60 s) while the cache warms up. This is expected behavior.