# Get Started The **Multimodal DataPrep microservice** builds and stores frame-level, image, and text embeddings in the configured vector database — VDMS by default, or Milvus — while preserving the raw assets in the configured object storage — MinIO by default, or the local filesystem. This guide explains how to launch the service, configure runtime options, and exercise the primary APIs. Backend selection is covered in [Pluggable Backends](./pluggable-backends.md); this walkthrough uses the default VDMS + MinIO stack. ## Configuration and Setup Multimodal DataPrep ships with Docker Compose manifests (`docker/compose*.yaml`) for different backends — for example `docker/compose.yaml` provisions the default MinIO + VDMS Vector DB + DataPrep stack, while `docker/compose-milvus.yaml` runs against Milvus. Always `source` the accompanying `setup.sh` script so the exported environment variables remain in your shell. ## Prerequisites Before you begin, ensure the following: - **System Requirements**: Verify that your system meets the [minimum requirements](./get-started/system-requirements.md). - **Docker Installed**: Install Docker. For installation instructions, see [Get Docker](https://docs.docker.com/get-docker/). This guide assumes basic familiarity with Docker commands and terminal usage. If you are new to Docker, see [Docker Documentation](https://docs.docker.com/) for an introduction. ## Environment Variables The table below lists the core configuration knobs. `setup.sh` seeds defaults, but you can override them before sourcing the script. The defaults below assume the **VDMS + MinIO** backends; to run against **Milvus** or **local filesystem** storage, set the backend selectors below and see [Pluggable Backends](./pluggable-backends.md) for the full backend-specific reference. | Variable | Required | Default | Purpose | | --- | --- | --- | --- | | `MM_DATAPREP_VECTORDB_BACKEND` | Optional | `vdms` | Active vector-database backend: `vdms` or `milvus`. | | `MM_DATAPREP_STORAGE_BACKEND` | Optional | `minio` | Active object-storage backend: `minio` or `local`. | | `MM_DATAPREP_MILVUS_URI` | Optional | _(none)_ | Full Milvus URI (e.g. `http://milvus:19530`). Overrides `MILVUS_HOST`/`MILVUS_PORT`. Used only when `VECTORDB_BACKEND=milvus`. | | `MM_DATAPREP_MILVUS_HOST` / `MM_DATAPREP_MILVUS_PORT` | Optional | _(none)_ / `19530` | Milvus host and port when `MILVUS_URI` is not set. Used only when `VECTORDB_BACKEND=milvus`. | | `MM_DATAPREP_VDB_METRIC_TYPE` | Optional | `IP` | Vector similarity metric applied to both VDMS and Milvus (e.g. `IP`, `L2`). | | `MM_DATAPREP_VDB_INDEX_TYPE` | Optional | `FLAT` | Vector index type for backends that require it (e.g. Milvus `FLAT`). | | `MM_DATAPREP_LOCAL_STORAGE_PATH` | Optional | `/tmp/dataprep/storage` | Root directory for the `local` storage backend; each bucket maps to a subdirectory. Used only when `STORAGE_BACKEND=local`. | | `MINIO_ROOT_USER`, `MINIO_ROOT_PASSWORD` | ✅ | _(none)_ | Credentials used to bootstrap MinIO. Required only when `STORAGE_BACKEND=minio`. | | `MM_DATAPREP_MINIO_ENDPOINT` | ✅ | `minio-server:9000` | Host:port string DataPrep uses to communicate with MinIO from inside the container. Required only when `STORAGE_BACKEND=minio`. | | `MM_DATAPREP_DEFAULT_BUCKET_NAME` | ✅ | `video-summary` (via `setup.sh`) | Destination bucket for uploaded media and generated manifests. When running alongside pipeline-manager, set it to the same bucket pipeline-manager uses. | | `MM_DATAPREP_VDMS_VDB_HOST` / `MM_DATAPREP_VDMS_VDB_PORT` | ✅ | `vdms-vector-db` / `55555` | Connection information for VDMS Vector DB. Used only when `VECTORDB_BACKEND=vdms`. | | `MM_DATAPREP_DB_COLLECTION` | ✅ | `video-rag-test` | Vector-database collection/index that stores embeddings and metadata (applies to both VDMS and Milvus). | | `MM_DATAPREP_EMBEDDING_MODEL_NAME` | ✅ | _(none)_ | Model identifier used by the in-process embedding pipeline (for example `CLIP/clip-vit-b-32` for multimodal or `QwenText/qwen3-embedding-0.6b` for text-only embeddings). | | `MM_DATAPREP_USE_OPENVINO` | Optional | `true` | Enables OpenVINO acceleration for embedding generation. Set `false` to stay on PyTorch. | | `MM_DATAPREP_EMBEDDING_DEVICE` | Optional | `CPU` | Device for the in-process embedding pipeline (`CPU`, `GPU`, or `NPU`). | | `MM_DATAPREP_DETECTION_DEVICE` | Optional | `CPU` | Device override for object detection execution (`CPU`, `GPU`, or `NPU`). | | `MM_DATAPREP_EMBEDDING_BATCH_SIZE` | Optional | `32` | Number of items sent per embedding batch. | | `MM_DATAPREP_MAX_PARALLEL_WORKERS` | Optional | _(auto)_ | Hard cap for parallel workers when auto-scaling is too aggressive for the host. | | `MM_DATAPREP_ALLOW_DUPLICATE_UPLOADS` | Optional | `true` | When `false`, an upload whose byte content is identical to an already-ingested video is rejected with `409 Conflict`. Detection is content-based (SHA-256) and applies to `/media/upload`, `/media/upload/batch`, `/media/ingest-dir`, `/media/process`, and `/media/process/batch` (per item, reported in the job status). | | `MM_DATAPREP_FRAME_INTERVAL` | Optional | `15` | Extract every Nth frame during video processing. | | `MM_DATAPREP_LIVE_STREAM_ENABLED` | Optional | `true` | Enables the live (RTSP) stream CRUD endpoints and their background workers. | | `MM_DATAPREP_LIVE_STREAM_DB_HOST` | Optional | `localhost` | PostgreSQL host for the live-stream registry. Registrations survive a restart. | | `MM_DATAPREP_LIVE_STREAM_DB_PORT` | Optional | `5432` | PostgreSQL port for the live-stream registry. | | `MM_DATAPREP_LIVE_STREAM_DB_NAME` | Optional | `dataprep` | PostgreSQL database holding the `live_streams` registry table. | | `MM_DATAPREP_LIVE_STREAM_DB_USER` | Optional | `postgres` | PostgreSQL user for the live-stream registry. | | `MM_DATAPREP_LIVE_STREAM_DB_PASSWORD` | Optional | `postgres` | PostgreSQL password for the live-stream registry. | | `MM_DATAPREP_LIVE_STREAM_BUCKET` | Optional | `live-streams` | Bucket (or local directory) holding recorded live segments. Live embeddings use this as their `bucket_name`. | | `MM_DATAPREP_LIVE_STREAM_MAX_CONCURRENT` | Optional | `8` | Maximum simultaneously ingesting streams. Further `POST /media/streams` calls return `503`. | | `MM_DATAPREP_LIVE_SEGMENT_DURATION_SECONDS` | Optional | `10` | Length of each recorded MP4 playback segment. | | `MM_DATAPREP_LIVE_STORE_SEGMENTS` | Optional | `true` | Record MP4 segments (remuxed, not re-encoded) for playback. | | `MM_DATAPREP_LIVE_BATCH_MAX_AGE_SECONDS` | Optional | `20` | Flush a partially filled frame batch from a live source once its oldest frame reaches this age. A live stream never reaches end-of-stream, so without this the pipeline waits for a full `MM_DATAPREP_VIDEO_EXTRACTION_BATCH_SIZE` batch before producing any embedding. `0` disables the timer. | | `MM_DATAPREP_LIVE_RECONNECT_INTERVAL_SECONDS` | Optional | `5` | Delay between reconnect attempts after a live source drops. | | `MM_DATAPREP_LIVE_RECONNECT_MAX_ATTEMPTS` | Optional | `10` | Reconnect budget before a stream is marked `error`. `0` retries forever. | | `MM_DATAPREP_LIVE_RECONNECT_WINDOW_SECONDS` | Optional | `60` | Healthy-session duration after which the reconnect budget resets. | | `MM_DATAPREP_LIVE_RETENTION_HOURS` | Optional | `0` | Retention window for live embeddings and media. `0` keeps everything forever — the index then grows without bound. | | `MM_DATAPREP_LIVE_RETENTION_SWEEP_MINUTES` | Optional | `15` | How often the retention sweeper runs when retention is enabled. | | `MM_DATAPREP_LIVE_CLOCK_CHECK_ENABLED` | Optional | `true` | On starting a live stream, probe the camera's HTTP `Date` header and log a warning when its clock is adrift from the host. Purely diagnostic — ingestion timestamps always come from the host clock. | | `MM_DATAPREP_LIVE_CLOCK_SKEW_WARN_SECONDS` | Optional | `2` | Camera-to-host drift, in seconds, above which the clock check warns. | | `MM_DATAPREP_ENABLE_OBJECT_DETECTION` | Optional | `true` | Toggles object-detection crop extraction. | | `MM_DATAPREP_DETECTION_MODEL_NAME` | ✅ (when detection is enabled) | _(none)_ | Hugging Face OpenVINO detection model repo, for example `OpenVINO/yolox_s-int8-ov`; `OpenVINO/yolox_tiny-fp16-ov`, `OpenVINO/yolox_l-int8-ov` and `OpenVINO/YOLO11n-fp16-ov` are also verified on CPU, GPU and NPU (see [Device selection](#device-selection-mm_dataprep_embedding_device-mm_dataprep_detection_device) and ). Only the `OpenVINO/` namespace is accepted, and the model must be an object-detection model supported by OpenVINO Model API. The image ships no detection models: only the selected one is downloaded on first use into `MM_DATAPREP_DETECTION_MODEL_DIR/` and reused from the volume afterwards. Each model keeps its own license (for example YOLOX is Apache-2.0, YOLO11 is AGPL-3.0); review the model card. | | `MM_DATAPREP_DETECTION_CLASSES` | Optional | _(empty)_ | Comma-separated class filter, for example `person,car`. Case-insensitive; spaces and hyphens are normalized to `_` (Hugging Face OpenVINO models use labels such as `traffic_light`, `cell_phone`). The tested OpenVINO detection models use the 80 [COCO classes](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/coco.yaml); the model's own labels are authoritative. Classes the model does not have make detector initialization fail, and the error lists the available classes. __Empty keeps all classes__. | | `MM_DATAPREP_DETECTION_MODEL_DIR` | Optional | `/app/models/object-detection` | Download/cache directory for detection models (backed by the `dataprep-detection-models` volume). | | `MM_DATAPREP_DETECTION_CONFIDENCE` | Optional | `0.85` | Minimum confidence threshold for detections. | | `MM_DATAPREP_ROI_CONSOLIDATION_ENABLED` | Optional | `false` | Enables ROI consolidation (merging overlapping detections). | | `MM_DATAPREP_ROI_CONSOLIDATION_IOU_THRESHOLD` | Optional | `0.2` | IoU threshold used to group overlapping boxes into a single ROI. | | `MM_DATAPREP_ROI_CONSOLIDATION_CLASS_AWARE` | Optional | `false` | Merge only boxes of the same class when `true`. | | `MM_DATAPREP_ROI_CONSOLIDATION_CONTEXT_SCALE` | Optional | `0.2` | Expands merged ROIs by this fraction of their width/height. | | `MM_DATAPREP_VIDEO_SHM_MAX_BLOCKS` | Optional | `512` | Shared memory block count for the video decode and embedding pipeline. | | `MM_DATAPREP_VIDEO_SHM_BLOCK_SIZE` | Optional | `6220800` | Per-block shared memory size in bytes (default sized for 1080p RGB frames). | | `MM_DATAPREP_VIDEO_EXTRACTION_BATCH_SIZE` | Optional | `256` | Decoder-side batch size used when extracting frames for processing. | | `MM_DATAPREP_PIPELINE_QUEUE_MAXSIZE` | Optional | `16` | Queue capacity for inter-stage pipeline buffers. | | `MM_DATAPREP_PIPELINE_COMPLETION_QUEUE_MAXSIZE` | Optional | `1` | Queue capacity for completion/result handoff stage. | | `MM_DATAPREP_DETECTION_WORKER_THREADS` | Optional | `2` | Local thread count used by object-detection worker stage. | | `MM_DATAPREP_EMBED_WORKER_THREADS` | Optional | `2` | Local thread count used by embedding worker stage. | | `MM_DATAPREP_PIPELINE_QUEUE_GET_TIMEOUT_S` | Optional | `1.0` | Timeout in seconds for pipeline queue reads before retry loops. | | `MM_DATAPREP_PIPELINE_STALL_TIMEOUT_S` | Optional | `300` | Fails a video ingestion when no pipeline stage makes progress for this many seconds (for example a detection or embedding inference hung by a GPU hang), instead of leaving the job `running` forever. `0` disables the watchdog. | | `MM_DATAPREP_SAVE_RUNTIME_PIPELINE_STATS` | Optional | `false` | Persist batch/stream runtime stats JSON artifacts for debugging and profiling. | | `MM_DATAPREP_ENABLE_TRACING` | Optional | `false` | Enables trace emission for decode/detect/embed/store stages. | | `MM_DATAPREP_VIDEO_FRAME_DECODER_WORKERS` | Optional | `2` | Number of decoder workers used in frame extraction utilities. | | `MM_DATAPREP_VIDEO_FRAME_LOG_LEVEL` | Optional | `INFO` | Log level for decoder internals (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`). | | `MM_DATAPREP_OV_MODELS_DIR` | Optional | `/app/ov_models` | Persistent mount that caches OpenVINO-optimized models. | | `MM_DATAPREP_METRICS_MANAGER_URL` | Optional | _(empty / disabled)_ | Metrics Manager base URL. When set, each completed video pipeline publishes `dataprep_embeddings_per_second` to `/api/v1/metrics/simple`. | | `MM_DATAPREP_METRICS_MANAGER_TIMEOUT_SECONDS` | Optional | `2.0` | Timeout for one Metrics Manager publish attempt. Publishing is asynchronous and never delays ingestion. | | `MM_DATAPREP_ALLOW_ORIGINS`, `MM_DATAPREP_ALLOW_METHODS`, `MM_DATAPREP_ALLOW_HEADERS` | Optional | `*` | CORS configuration applied by FastAPI. | ### Device selection (`MM_DATAPREP_EMBEDDING_DEVICE`, `MM_DATAPREP_DETECTION_DEVICE`) DataPrep configures its two compute stages independently — there is no baseline device. Each variable defaults to `CPU` when unset: - `MM_DATAPREP_EMBEDDING_DEVICE` — device for the in-process embedding pipeline. - `MM_DATAPREP_DETECTION_DEVICE` — device for object detection. > [!IMPORTANT] > These variables are read directly by the DataPrep container. You > can `source` a setup script (which exports the `CPU` defaults) or set them > explicitly before running `docker compose up`. Examples (run before sourcing the setup script): ```bash # Offload detection to NPU and embedding to GPU (independent per-component devices) export MM_DATAPREP_DETECTION_DEVICE=NPU export MM_DATAPREP_EMBEDDING_DEVICE=GPU # Run both stages on GPU export MM_DATAPREP_EMBEDDING_DEVICE=GPU export MM_DATAPREP_DETECTION_DEVICE=GPU # Embedding on GPU, but keep detection on CPU export MM_DATAPREP_EMBEDDING_DEVICE=GPU export MM_DATAPREP_DETECTION_DEVICE=CPU ``` When targeting `NPU`, confirm the selected model supports NPU inference via the [OpenVINO Supported Models](https://docs.openvino.ai/2026/documentation/compatibility-and-support/supported-models.html) page. For detection, these models have been verified on CPU, GPU and NPU: | Model | License | Notes | |---|---|---| | `OpenVINO/yolox_s-int8-ov` | Apache-2.0 | Recommended default; balanced accuracy and speed, fastest on CPU and NPU | | `OpenVINO/yolox_tiny-fp16-ov` | Apache-2.0 | Lightest; fastest on GPU | | `OpenVINO/yolox_l-int8-ov` | Apache-2.0 | Higher accuracy (more and smaller objects) at a higher compute cost; best on GPU or NPU | | `OpenVINO/YOLO11n-fp16-ov` | AGPL-3.0 | Small YOLO11 model; review the AGPL-3.0 terms before use | On GPU, the FP16 variants of YOLOX (for example `OpenVINO/yolox_s-fp16-ov`) run faster than the INT8 variants. Verify other models on your target device before relying on them. > **Running everything on NPU:** Setting both embedding and detection to `NPU` > (via `MM_DATAPREP_EMBEDDING_DEVICE=NPU` and `MM_DATAPREP_DETECTION_DEVICE=NPU`) is > functionally supported — both stages run on NPU through OpenVINO. However, the > host has a single NPU, so the embedding and detection stages contend for the > same accelerator. It works, but it is not optimal for throughput. For best > performance, split the load across accelerators (for example, keep embedding on > `NPU` and detection on `GPU`/`CPU`, or vice versa). ### Advanced tuning Additional environment variables are available for high-throughput scenarios: - `MM_DATAPREP_MAX_PARALLEL_WORKERS` — hard cap on worker threads (auto-calculated when unset). - `MM_DATAPREP_OV_PERFORMANCE_MODE`, `OV_PERFORMANCE_HINT_NUM_REQUESTS`, `OV_NUM_STREAMS` — forward performance hints to OpenVINO when running on CPU or GPU. - `MM_DATAPREP_VIDEO_SHM_MAX_BLOCKS`, `MM_DATAPREP_VIDEO_SHM_BLOCK_SIZE` — tune shared-memory capacity for frame transport. - `MM_DATAPREP_VIDEO_EXTRACTION_BATCH_SIZE`, `MM_DATAPREP_PIPELINE_QUEUE_MAXSIZE`, `MM_DATAPREP_PIPELINE_QUEUE_GET_TIMEOUT_S` — tune decode and queue backpressure behavior. - `MM_DATAPREP_DETECTION_WORKER_THREADS`, `MM_DATAPREP_EMBED_WORKER_THREADS` — tune stage-local worker counts. - `MM_DATAPREP_SAVE_RUNTIME_PIPELINE_STATS`, `MM_DATAPREP_ENABLE_TRACING`, `MM_DATAPREP_VIDEO_FRAME_LOG_LEVEL` — enable diagnostics and control verbosity. Export overrides before sourcing the setup script: ```bash export EMBEDDING_MODEL_NAME="CLIP/clip-vit-b-16" export MINIO_ROOT_USER="minioadmin" export MINIO_ROOT_PASSWORD="minioadmin" export MM_DATAPREP_EMBEDDING_DEVICE="CPU" export MM_DATAPREP_DETECTION_DEVICE="CPU" export MM_DATAPREP_DETECTION_MODEL_NAME="OpenVINO/yolox_s-int8-ov" source ./setup.sh --nosetup ``` > [!TIP] > When you only need long-form text embeddings—such as the combined `--all` mode in the video search and summarization sample—set `EMBEDDING_MODEL_NAME="QwenText/qwen3-embedding-0.6b"` before sourcing `setup.sh`. The script forwards this value to the DataPrep container as `MM_DATAPREP_EMBEDDING_MODEL_NAME`, enabling Qwen-backed text embeddings without any additional flags. ## ROI consolidation (optional) ROI consolidation merges overlapping detections into a single crop and optionally expands that crop for more context. This can reduce duplicate crops and improve embedding coverage when multiple detections overlap the same object. To enable it: ```bash export MM_DATAPREP_ROI_CONSOLIDATION_ENABLED=true ``` The following environment variables control ROI consolidation: - `MM_DATAPREP_ROI_CONSOLIDATION_ENABLED`: Master switch for ROI consolidation logic. - `MM_DATAPREP_ROI_CONSOLIDATION_IOU_THRESHOLD`: IoU threshold used to cluster overlapping boxes. IoU is $\frac{\text{intersection area}}{\text{union area}}$ for two boxes; higher values mean only tighter overlaps merge, lower values merge more aggressively. - `MM_DATAPREP_ROI_CONSOLIDATION_CLASS_AWARE`: When `true`, only boxes of the same class can be merged. When `false`, overlapping boxes across classes can merge (useful for mixed-class clusters). - `MM_DATAPREP_ROI_CONSOLIDATION_CONTEXT_SCALE`: Expand merged ROI by this fraction of its size. Higher values include more surrounding context; lower values keep crops tighter to the merged box. Use `source ./setup.sh --conf` to print the resolved Docker Compose configuration with your overrides applied. ## Quick Start with Docker > [!IMPORTANT] > Do not run `docker build` directly against `docker/Dockerfile` from the `multimodal-dataprep` directory. Always execute `./build.sh` so the build uses the `microservices/` context and includes the local `multimodal-embedding-serving` source dependency. The user has an option to either [build the docker images](./get-started/build-from-source.md#steps-to-build) or use prebuilt images as documented below. **Configure the registry**: The Multimodal DataPrep microservice uses the registry URL and tag to pull the required image. ```bash export REGISTRY_URL=intel export TAG=2026.3.0-rc1 ``` 1. **Clone the repository and enter the project.** ```bash git clone https://github.com/open-edge-platform/edge-ai-libraries.git -b release-2026.3.0 cd edge-ai-libraries/microservices/visual-data-preparation-for-retrieval/multimodal-dataprep ``` 2. **Export required secrets and model selection.** ```bash export MINIO_ROOT_USER="minioadmin" export MINIO_ROOT_PASSWORD="minioadmin" export EMBEDDING_MODEL_NAME="CLIP/clip-vit-b-32" export MM_DATAPREP_DETECTION_MODEL_NAME="OpenVINO/yolox_s-int8-ov" # any https://huggingface.co/OpenVINO detection model # Also verified on CPU/GPU/NPU: OpenVINO/yolox_tiny-fp16-ov (lightest), OpenVINO/yolox_l-int8-ov (more accurate), # OpenVINO/YOLO11n-fp16-ov (AGPL-3.0) # Optional: keep only some classes (COCO class names: https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/coco.yaml) # export MM_DATAPREP_DETECTION_CLASSES="person,car" ``` Object detection has no default model. Set `MM_DATAPREP_DETECTION_MODEL_NAME`, or set `MM_DATAPREP_ENABLE_OBJECT_DETECTION=false` to skip object crops. The selected model is downloaded from Hugging Face on first start; for offline hosts, pre-populate the `dataprep-detection-models` volume (or set `HF_HUB_OFFLINE=1` with a populated cache directory). For text-only scenarios replace the `EMBEDDING_MODEL_NAME` line with: ```bash export EMBEDDING_MODEL_NAME="QwenText/qwen3-embedding-0.6b" ``` 3. **Start the stack.** Run `source ./setup.sh` to export the environment variables, then start the default stack (MinIO, VDMS, and DataPrep) with `docker compose -f docker/compose.yaml up -d --build`. To run against Milvus instead, use `docker compose -f docker/compose-milvus.yaml up -d --build`. 4. **Confirm the stack is healthy.** ```bash docker ps --format "table {{.Names}}\t{{.Status}}" ``` 5. **Open the interactive docs.** Navigate to `http://localhost:6007/docs` (adjust if you changed `MM_DATAPREP_HOST_PORT`) to view the OpenAPI schema. 6. **Shut everything down when finished.** Use `source ./setup.sh --down` (or `docker compose -f docker/compose.yaml down`) to stop services. ## Usage The FastAPI application is mounted under `/v1/dataprep`. ### Health probe ```bash curl http://localhost:6007/v1/dataprep/health ``` Health responses include the embedding client preload status, model name, and device. ### Upload and process a new video ```bash curl -X POST "http://localhost:6007/v1/dataprep/media/upload" \ -H "Content-Type: multipart/form-data" \ -F "file=@/path/to/video.mp4" \ -F "frame_interval=10" \ -F "enable_object_detection=true" \ -F "tags=intersection" -F "tags=night" ``` The service streams the asset to the configured object storage, extracts frames (and crops), generates embeddings, and persists metadata in the configured vector database. ### Upload and process an image The same endpoint accepts images (`.jpg`, `.jpeg`, `.png`, `.webp`, `.bmp`, `.gif`). Images are embedded directly (full image plus optional detected-object crops) into the same vector collection as video frames, enabling cross-modal search. ```bash # Multipart file upload curl -X POST "http://localhost:6007/v1/dataprep/media/upload?enable_object_detection=true" \ -F "file=@/path/to/image.jpg" -F "tags=cat" # Inline base64 (data URL accepted) via the JSON typed-source endpoint curl -X POST "http://localhost:6007/v1/dataprep/media/ingest" \ -H "Content-Type: application/json" \ -d '{"type": "image_base64", "image_base64": "data:image/png;base64,iVBORw0KGgo..."}' # Remote URL (server downloads, size-capped at 50 MB) curl -X POST "http://localhost:6007/v1/dataprep/media/ingest" \ -H "Content-Type: application/json" \ -d '{"type": "image_url", "image_url": "https://example.com/cat.jpg"}' ``` ### Process an existing video in MinIO ```bash curl -X POST "http://localhost:6007/v1/dataprep/media/process" \ -H "Content-Type: application/json" \ -d '{ "bucket_name": "video-summary", "video_id": "traffic_cam_2024_10_21", "frame_interval": 12, "enable_object_detection": true, "tags": ["traffic", "daytime"] }' ``` ### Attach a human-authored summary To attach a human-authored summary to a video, use this command: ```bash curl -X POST "http://localhost:6007/v1/dataprep/summary" \ -H "Content-Type: application/json" \ -d '{ "bucket_name": "video-summary", "video_id": "traffic_cam_2024_10_21", "video_summary": "Vehicle stopped at intersection for 45 seconds", "video_start_time": 12.5, "video_end_time": 57.0, "tags": ["summary", "manual"] }' ``` ### Discover, download, and delete content You can use the following commands to discover, download, and delete content: ```bash # List processed videos (video_id + filenames) curl "http://localhost:6007/v1/dataprep/media" # Download a processed clip (stream or attachment) curl -L "http://localhost:6007/v1/dataprep/media/download?video_id=traffic_cam_2024_10_21" -o clip.mp4 # Delete a video (removes its storage object(s) + vector embeddings) curl -X DELETE "http://localhost:6007/v1/dataprep/media/my-bucket/traffic_cam_2024_10_21" ``` ### Review processing telemetry The telemetry endpoint captures per-request wall-clock timings, stage durations, throughput, and batch-level stats. Query the most recent entries directly from the DataPrep service (or via the pipeline-manager proxy) with: ```bash curl --location 'http://localhost:6016/telemetry?limit=5' ``` See the [Telemetry Metrics](telemetry-metrics.md) reference for a complete breakdown of every field and how each value is calculated. ## Validate Services 1. Call `GET /v1/dataprep/health` – expect `status: ok`, the embedding client status, model name, device, and OpenVINO flag. 2. Upload a small MP4 via `/media/upload` and confirm: - The response payload reports `success`. - `GET /v1/dataprep/media` lists the generated `video_id` and manifests. - The MinIO console (`http://localhost:6011`) shows the raw asset, thumbnails, and crops. 3. Inspect the vector database to verify entries in the `video-rag-test` collection (for the default VDMS backend, use `vdms_cli` or a custom client; for Milvus, use a Milvus client such as `pymilvus` or Attu). ## Supporting Resources - [Overview](index.md) - [System Requirements](./get-started/system-requirements.md) - [Architecture Overview](./how-it-works-architecture.md) - [Media Ingestion Flow](./how-it-works-ingestion-flow.md) - Detailed flow diagrams of the video and image processing pipelines - [API Reference](./api-reference.md) - [Troubleshooting](./troubleshooting.md) :::{toctree} :hidden: ./get-started/system-requirements.md ./get-started/build-from-source.md :::