Release Notes: Video Search and Summarization Sample Application#

Version 2026.2.0#

Release Date: August 4, 2026

New:

Backend-agnostic vector search: Select the vector database with VECTORDB_BACKEND (vdms default, milvus). All vector similarity search is delegated to the standalone vector-retriever microservice; video-search keeps query orchestration and frame-to-video aggregation, so results are identical in shape across backends.
Multimodal DataPrep: Replaced the legacy VDMS DataPrep with the multimodal-dataprep microservice across Docker Compose, Helm, and documentation.
Metrics Manager integration: Replaced the legacy telemetry collector; the search UI now shows live multi-GPU telemetry (ENABLE_METRICS_MANAGER=true).
Batch video upload and embedding: Multi-select upload in the UI uploads all files and submits a single asynchronous batch embeddings job, with new search-embeddings-batch and search-embeddings-jobs Pipeline Manager endpoints and job-status polling. Single-file upload keeps the synchronous path.
Directory watcher batch processing: The directory watcher service now batches video uploads and embedding creation.
Duplicate-content upload policy: New MM_DATAPREP_ALLOW_DUPLICATE_UPLOADS setting (Docker Compose and Helm global.env) rejects content-identical re-uploads with HTTP 409. Enforced per item for batch-processed media, so a duplicate fails only its own item.
Model download support: Models are now fetched by a dedicated model-download step in setup.sh, including generic YOLO ids from the ultralytics hub for object detection.
Search score transparency: Search results show the query-normalized relevance score alongside the underlying raw segment score and peak frame similarity, with a per-stage scoring breakdown popover and a weak-match indicator.
Makefile, functional test suite, and CI security scans for the Video Search and Summarization sample application.
AI agent skills: Added VSS agent skills (deploy, build, troubleshoot, summarize, search, and more) for AI coding assistants.

Improved:

Search orchestration alignment for accelerator usage: Updated setup and compose behavior for Video Search so NPU device selections are preserved and propagated consistently for DataPrep and multimodal embedding services.
Simplified per-component device model (Compose): Retired the redundant VDMS_DATAPREP_DEVICE baseline knob. Device selection is now purely per-component — DATAPREP_EMBEDDING_DEVICE, DATAPREP_DETECTION_DEVICE, and MME_EMBEDDING_DEVICE (each defaults to CPU) — matching the Helm chart model. ENABLE_EMBEDDING_GPU is now a mode-aware embedding shortcut (sdk→DataPrep embedding GPU, api→MME embedding GPU).
Helm accelerator support for search stack: Updated VSS Helm subcharts for multimodal-embedding-ms and multimodal-dataprep to support NPU as an accelerator path (device-key validation, resource requests/limits, and /dev/accel mounts).
Helm accelerator device permissions: Added global.accelGroupIds so the host gids owning /dev/dri (GPU) and /dev/accel (NPU) are injected into the pod supplementalGroups, letting the non-root container open the accelerator device (mirrors the Compose group_add render/video groups). Fixes NPU/GPU device initialization falling back to CPU-only.
Helm OpenVINO™ model cache: Added a persistent OpenVINO™ cache (ovCacheDir, default /app/ov_models/ov_cache) for multimodal-embedding-ms and multimodal-dataprep, plus a longer DataPrep startupProbe budget, so GPU/NPU model compilation completes once and is reused across pod restarts instead of recompiling (avoids startup crash loops).
Helm single-source image override: global.registry, global.tag, and global.pullPolicy now apply across all VSS service images (pipeline-manager, video-ingestion, video-search, vss-ui, multimodal-dataprep, multimodal-embedding-serving) from one place, with independent per-service PVCs for model/cache data.
Helm value override precedence: Subchart defaults no longer shadow global.env overrides for MM_DATAPREP_ALLOW_DUPLICATE_UPLOADS and SDK_USE_OPENVINO, so --set global.env.<KEY> now takes effect.
Clearer embedding error reporting: The video embedding flow now surfaces the real upstream DataPrep error instead of a misleading “Request timed out” message; only genuine timeouts (408/504/connection aborts) are reported as timeouts.
Graceful OVMS cache fallback: setup.sh now degrades cleanly when GPU drivers are absent instead of failing model cache setup.
Original filenames in embedding metadata: The uploaded video name is threaded end-to-end into stored embedding metadata, so search results show the original filename.
Service health and startup dependencies hardened across the stack, with retries for transient batch job polling failures.
Search deployment documentation refresh: Added a dedicated Deployment Options for Video Search matrix (SDK/API with CPU/GPU/NPU combinations), including explicit DATAPREP_EMBEDDING_DEVICE, MME_EMBEDDING_DEVICE, and DATAPREP_DETECTION_DEVICE examples for accelerator-specific routing.
Helm user-guide clarifications: Updated Helm guidance to include NPU device/key combinations and matching-device recommendations for shared PVC scheduling.
Documentation: Aligned user and agent documentation with the multi-backend architecture, refreshed the DataPrep, retriever, and VSS OpenAPI specs, added 4K/8K video ingestion troubleshooting guidance, and split 2025 release notes into a separate document.
Maintenance: Refreshed dependency version constraints and lock files, pinned Dockerfiles to release versions, aligned the DataPrep bucket with the video summary bucket, updated the VSS health check with a manual CI workflow trigger, and removed the profile_dataprep script and the vestigial EMBEDDING_PROCESSING_MODE chart value.

Fixed:

Duplicate-upload conflicts no longer leave orphaned video tiles in the UI.
Search result tiles no longer render as black thumbnails.
Assorted search-ms and video embedding flow defects.

Version 2026.1.0#

Release Date: June 17, 2026

New:

EXPERIMENTAL - vLLM Intel® Arc™ Pro B-series GPU Support: Added initial experimental support for running vLLM on Intel® Arc™ Pro B-series GPUs (XPUs) via new ENABLE_VLLM_GPU environment variable and docker/compose.vllm.xpu.yaml Docker Compose overlay. This feature enables GPU-accelerated VLM captioning and LLM summarization on Intel® Arc™ Pro B-series hardware (e.g., B60, B65, B70).

Note: This is an experimental feature in early stages and may require additional tuning and optimization. Performance characteristics are still being evaluated. Not recommended for production use.
Added a a new Dual UI mode with a new --summary --search CLI argument for setup.sh that allows running both the summary and the search applications simultaneously at /summary and /search URI endpoints respectively.
Added Dual UI support for Helm chart installations by allowing a values override file to be provided for summary and search modes simultaneously.

Improved:

Updated setup script and nginx configuration files to allow flexible UI routing for each existing mode of deployment (summary mode, search mode, Unified UI Mode) and the new Dual UI mode.
Refactored Helm chart to use a reusable vssui subchart with multi-mode nginx and consolidated embedding model config under global.embeddingModelName.
Updated DL Streamer base image to 2026.1.0-ubuntu24-rc1 for Video Ingestion Microservice.
Setup Script: Updated the environment variable to setup embedding models. New MULTIMODAL_EMBEDDING_MODEL and the existing TEXT_EMBEDDING_MODEL are used to provide embedding models in relevant modes.
Docker Compose: Replaced curl with Python urllib package in the container healthcheck command for a lighter runtime footprint for Audio Analyzer.
Docker Compose: Replaced environment variables with hard-coded mount paths. This helps in stopping containers without looking for preset variables.
Build Script: Removed Audio-Analyzer from the dependency build pipeline. A frozen version 1.3.3 will be used for the Audio Analyzer microservice for the current and all subsequent releases.
Setup Script: Removed unused environment variables and several environment variables being used as mount directories in Docker Compose files.

Version 1.3.3-rc1#

Release Date: 05 May 2026

Features:

Configurable final video summary: Added PM_PRODUCE_FINAL_SUMMARY feature flag to make the final LLM map-reduce video summary optional. When disabled, chunk-wise summaries are displayed chronologically instead. A per-video UI override checkbox is available in both upload flows. Audio transcript summarization is automatically skipped when the final summary is turned off.
Audio transcript summarization: Added audio transcript summarization support and improved audio transcription accuracy.
OVMS-first architecture: Replaced the standalone vlm-openvino-serving microservice with OpenVINO™ Model Server (OVMS) as the unified inference backend for both VLM captioning and LLM summarization. This is a breaking change; the vlm-inference subchart and container have been removed.
Performance Optimizations (MME & VDMS-Data-Prep):
- Refactored pre-processing and inference with AsyncInferQueue based OpenVINO™ inference and static shape model compilation for iGPU.
- Added ThreadPool for parallel open_clip image pre-processing with support for input tensor batching and padding for optimal OpenVINO™ inference paths.
- Introduced PyAV-based video decode abstraction supporting keyframes and uniform sampled frames extraction with producer-consumer pattern for parallel decode and frame translation to PIL.
- Enabled multiple/parallel decoder instances for file, RTSP stream, and bytes input sources.
- Implemented frame batching for pipelined pre-processing and inference with integrated PyAV decoder in VDMS data-prep.
Search Timeout and Resource Management: Added SEARCH_DATAPREP_TIMEOUT_MS configuration to prevent VSS-UI timing out during embedding creation. Added ulimit constraints with soft and hard limits to enable shared memory creation and define memory block allocation boundaries.

HW used for validation:

Intel® Xeon® 5 + Intel® Arc™ B580 GPU
Vanilla Kubernetes Cluster

Known Issues/Limitations:

This release includes only limited testing on EMT‑S and EMT‑D, some behaviors may not yet be fully validated across all scenarios.
HW sizing of the Video Search or Video Summarization pipeline is in progress. Optimization of the pipelines will follow HW sizing.
Known issues are internally tracked. Reference not provided here.
how-to-performance document is not updated yet. HW sizing details will be added to this section shortly.
NPU support with OVMS is added as experimental feature and may not work for all models or configurations.

Version 1.3.2#

Release Date: 17 Feb 2026