Troubleshooting#
Common issues#
Startup fails with “model name must be provided”: Set
EMBEDDING_MODEL_NAMEbefore launching Docker (required for both SDK and API modes).Startup fails with “Object detection is enabled but no detection model is configured”: Set
MM_DATAPREP_DETECTION_MODEL_NAMEto a Hugging Face OpenVINO detection model repo (for exampleOpenVINO/yolox_s-int8-ov), or setMM_DATAPREP_ENABLE_OBJECT_DETECTION=false. A value outside theOpenVINO/namespace fails with “MM_DATAPREP_DETECTION_MODEL_NAME must be a Hugging Face model repo in the ‘OpenVINO’ namespace (OpenVINO/), got: …”. Object detection disabled unexpectedly (
detection_status: "not_loaded"inGET /v1/dataprep/health): The service keeps running without object crops when the detector cannot initialize; the logs explain why. Common causes:Hugging Face download failed – the selected model is downloaded on first use into
MM_DATAPREP_DETECTION_MODEL_DIR/<repo id>. Ensure the host can reachhuggingface.co(sethttp_proxy/https_proxy/no_proxybehind a proxy) and that theDETECTION_MODELS_VOLUME_NAMEvolume (defaultdataprep-detection-models) exists. For offline hosts, pre-populate the volume with the model’s*.xml/*.binfiles (or setHF_HUB_OFFLINE=1with a populated cache directory).Unknown class in
MM_DATAPREP_DETECTION_CLASSES– every listed class must exist in the model’s labels, otherwise the log reports “MM_DATAPREP_DETECTION_CLASSES contains classes not provided by: … Available classes: …”. Hugging Face OpenVINO models use underscores (for example traffic_light,cell_phone).Not a detection model – the repo must load as an OpenVINO Model API
DetectionModel; otherwise the log reports “is not an object detection model”. Unsupported device – on NPU, check per-model support on the OpenVINO Supported Models page and test the model; some architectures return no detections on NPU.
API mode returns 502: Verify the multimodal embedding service is healthy at
MULTIMODAL_EMBEDDING_ENDPOINT(seedocker compose -f docker/compose-with-embedding.yaml ps).Uploads rejected: Files larger than 500 MB are not accepted by the FastAPI upload endpoint. Stage the video directly in MinIO and use
/media/processinstead.GPU acceleration inactive: Confirm
/dev/dri/*is mapped into the container, set the relevant device variable (MM_DATAPREP_EMBEDDING_DEVICEorMM_DATAPREP_DETECTION_DEVICE) toGPU, and keepMM_DATAPREP_USE_OPENVINO=true.NPU acceleration inactive: Confirm
/dev/accel/accel0is available on the host and mapped into the container, set the relevant device variable (MM_DATAPREP_EMBEDDING_DEVICEorMM_DATAPREP_DETECTION_DEVICE) toNPU, and keepMM_DATAPREP_USE_OPENVINO=true. Verify the selected model supports NPU inference via the OpenVINO Supported Models page.First NPU run is slow (one-time model compilation): The first time a model runs on NPU, OpenVINO compiles it to an NPU-specific blob, which takes noticeably longer than CPU/GPU startup. This is expected and happens once per model/configuration. The compiled blob is cached on the
OV_MODELS_DIRmount (default/app/ov_models), so subsequent runs reuse it and start quickly — persist this volume to retain the cache across container restarts.
Ingestion fails with “Pipeline made no progress”#
Symptom. A video ingestion request, or a batch job item, fails with an error such as:
Pipeline made no progress for 300s while waiting for detection/embedding/storage
to finish (MM_DATAPREP_PIPELINE_STALL_TIMEOUT_S=300). A detection or embedding inference
call is likely hung (for example a GPU hang); check device health or run
detection/embedding on CPU.
Why it happens. A stall watchdog fails the job when no pipeline stage hands
an item to the next one for MM_DATAPREP_PIPELINE_STALL_TIMEOUT_S seconds
(default 300). This usually means a detection or embedding inference call
never returned, for example after a GPU hang. Without the watchdog the job would
stay running forever.
Fix.
Check the accelerator: look for GPU hang or reset messages in
dmesg, and restart the container to recover the device.If it keeps happening on one device, run the affected stage on CPU with
MM_DATAPREP_DETECTION_DEVICE=CPUorMM_DATAPREP_EMBEDDING_DEVICE=CPU.If very slow hardware legitimately needs longer between stages, raise
MM_DATAPREP_PIPELINE_STALL_TIMEOUT_S.0disables the watchdog.
Retry the ingestion after the fix.