Get Started Guide#
Time to Complete: 30 mins
Programming Language: Python
Get Started#
Prerequisites#
Install Docker: Installation Guide.
Install Docker Compose: Installation Guide.
Install Intel Client GPU driver: Installation Guide.
Step 1: Get the docker images#
Option 1: Build from source#
Go to the target directory of your choice and clone the suite.
If you want to clone a specific release branch, replace main with the desired tag.
To learn more on partial cloning, check the Repository Cloning guide.
git clone --filter=blob:none --branch main https://github.com/open-edge-platform/metro-ai-suite.git
cd metro-ai-suite
Run the commands to build images for the microservices:
git clone https://github.com/open-edge-platform/edge-ai-libraries.git -b main
cd edge-ai-libraries/microservices
cd visual-data-preparation-for-retrieval/multimodal-dataprep
./build.sh
cd ../../
docker build -t vector-retriever-milvus:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg no_proxy=$no_proxy -f vector-retriever/milvus/src/Dockerfile .
cd multimodal-embedding-serving
docker build -t multimodal-embedding-serving:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg no_proxy=$no_proxy -f docker/Dockerfile .
cd ../../..
VLM OpenVINO Serving is not built from source. The Compose file references the published image by its pinned tag, so Compose pulls it automatically. To fetch it ahead of time:
docker pull intel/vlm-openvino-serving:2026.2.0
Run the command to build image for the application:
docker build -t visual-search-qa-app:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg no_proxy=$no_proxy -f visual-search-question-and-answering/src/Dockerfile .
Option 2: use remote prebuilt images#
Set a remote registry by exporting environment variables:
export REGISTRY="intel/"
export TAG="latest"
Step 2: Prepare host directories for models and data#
mkdir -p $HOME/data
If you would like to test the application with a demo dataset, please continue and follow the instructions in the Try with a demo dataset section later in this guide.
Otherwise, if you would like to use your own data (images and video), put it all in a directory on the host and point HOST_DATA_PATH at it before deploying the services, for example export HOST_DATA_PATH="$HOME/data". It defaults to $HOME/data.
Note
Supported media types are jpg, png, and mp4.
Step 3: Deploy#
Option1 (Recommended): Deploy with docker compose#
Go to the deployment files
cd visual-search-question-and-answering/ cd deployment/docker-compose/
Set up environment variables.
Note
You need to set models first.
Ubuntu:
export EMBEDDING_MODEL_NAME="CLIP/clip-vit-h-14" # Replace with other models if needed export VLM_MODEL_NAME="Qwen/Qwen2.5-VL-7B-Instruct" # Replace with other models if needed source env.sh
Important
You must set
EMBEDDING_MODEL_NAMEandVLM_MODEL_NAMEbefore runningenv.sh. See Supported models for Multimodal Embedding Serving for available embedding models, and Supported models for VLM OpenVINO for available VLM models.env.shonly sets a variable when it is not already exported, so anything you export beforehand wins. Besides the two model names, the settings most often changed are:Variable
Purpose
Default
HOST_DATA_PATHHost directory holding the media to index, mounted into the services
$HOME/dataHF_ENDPOINTHugging Face endpoint used to download models
https://hf-mirror.comVLM_COMPRESSION_WEIGHT_FORMATVLM weight precision, for example
int4to reduce memoryint8Important
HF_ENDPOINTdefaults to a mirror intended for mainland China. Setexport HF_ENDPOINT="https://huggingface.co"to download models from Hugging Face directly.Each microservice picks its inference device independently, so that bulk ingestion and interactive queries do not have to share one accelerator:
Variable
Microservice
Default
VLM_DEVICEvlm-openvino-serving, question answeringGPUEMBEDDING_DEVICEmultimodal-embedding-serving, query embeddingCPUDATAPREP_EMBEDDING_DEVICEmultimodal-dataprep, ingest embeddingGPUDATAPREP_DETECTION_DEVICEmultimodal-dataprep, object detectionGPUEach accepts
CPU,GPUorNPU. The ingest pipeline is the throughput bottleneck, so it takes the GPU by default, while query embedding is a single short request per search and runs on the CPU. Avoid placing the embedding model on the same GPU for both services unless it has memory to spare: two copies of a large model, such asCLIP/clip-vit-h-14, can exhaust the device and the next query then fails withCL_OUT_OF_RESOURCES.GPUselects the first GPU that OpenVINO reports. On a platform with an integrated GPU asGPU.0and a discrete GPU asGPU.1, name the one you want explicitly. Refer to OpenVINO’s query device sample to list the available indices. For example, to run everything on the CPU:export VLM_DEVICE=CPU export EMBEDDING_DEVICE=CPU export DATAPREP_EMBEDDING_DEVICE=CPU export DATAPREP_DETECTION_DEVICE=CPU
Note
MM_DATAPREP_EMBEDDING_DEVICEandMM_DATAPREP_DETECTION_DEVICEare the previous names of the two dataprep variables and are still honoured, so existing scripts keep working.DATAPREP_*takes precedence if both are set.Milvus/MinIO/etcd state now uses Docker named volumes by default. For cleanup/reset, use
source env.sh --clean-data(this preservesov-modelsanddataprep-detection-models).Object detection is disabled by default, so only full frames are indexed. To also embed detected object crops, set
DATA_INGEST_WITH_DETECT=trueandDATAPREP_DETECTION_MODELbefore runningenv.sh.DATAPREP_DETECTION_MODELtakes a Hugging Face OpenVINO object-detection model inOpenVINO/<repo>form, for exampleOpenVINO/yolox_s-int8-ov. It has no default, so ingestion fails if detection is enabled without it. The model is downloaded on first use and cached in thedataprep-detection-modelsvolume; licenses differ per model, so review the model card before enabling it. Optionally setDATAPREP_DETECTION_CLASSESto a comma-separated allowlist such asperson,carto keep only those classes.EMT-S:
If you are on an EMT-S platform, set up the variables correspondingly by running:
cd emt-s # go to emt-s specific files export EMBEDDING_MODEL_NAME="CLIP/clip-vit-h-14" # Replace with other models if needed export VLM_MODEL_NAME="Qwen/Qwen2.5-VL-7B-Instruct" # Replace with other models if needed source env.sh
Deploy with docker compose
docker compose -f compose_milvus.yaml up -d
It might take a while to start the services for the first time, as there are some models to be prepared.
Check if all microservices are up and runnning with
docker compose -f compose_milvus.yaml ps
Output:
NAME COMMAND SERVICE STATUS PORTS multimodal-dataprep "uvicorn src.main:ap…" multimodal-dataprep running (healthy) 0.0.0.0:9990->8000/tcp, :::9990->8000/tcp milvus-etcd "etcd -advertise-cli…" milvus-etcd running (healthy) 2379-2380/tcp milvus-minio "/usr/bin/docker-ent…" milvus-minio running (healthy) 0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp milvus-standalone "/tini -- milvus run…" milvus-standalone running (healthy) 0.0.0.0:9091->9091/tcp, 0.0.0.0:19530->19530/tcp, :::9091->9091/tcp, :::19530->19530/tcp multimodal-embedding gunicorn -b 0.0.0.0:8000 - ... Up (unhealthy) 0.0.0.0:9777->8000/tcp,:::9777->8000/tcp retriever-milvus "uvicorn retriever_s…" retriever-milvus running (healthy) 0.0.0.0:7770->7770/tcp, :::7770->7770/tcp visual-search-qa-app "streamlit run app.p…" visual-search-qa-app running (healthy) 0.0.0.0:17580->17580/tcp, :::17580->17580/tcp vlm-openvino-serving "/bin/bash -c '/app/…" vlm-openvino-serving running (healthy) 0.0.0.0:9764->8000/tcp, :::9764->8000/tcp
Option2: Deploy in Kubernetes#
Refer to Deploy with helm for details.
Try with a demo dataset#
*Applicable to deployment with Option 1 (docker compose deployment).
Prepare demo dataset DAVIS#
Use the prepare_demo_dataset.sh script shipped in tests/. It runs on the host
(it needs curl, unzip and ffmpeg) and writes the dataset into the directory
shared with the dataprep microservice:
bash tests/prepare_demo_dataset.sh
Check your host data directory $HOME/data (or $HOST_DATA_PATH) and see if
DAVIS is there. To save time, only a subset of the dataset is prepared: it is
stored in $HOME/data/DAVIS/subset, half as videos and half as sampled images.
Use this path in the next step.
The script also writes a meta/<basename>.json sidecar per item with a camera
and a capture_date field, so the camera and date filters in the UI have
values to filter on.
Use it on Web UI#
Go to http://{host_ip}:17580 with a browser. Put the exact path to the subset of demo dataset (usually/home/user/data/DAVIS/subset, may vary according to your local username) into file directory on host. Click UpdataDB and wait for the uploading done.
Try searching with query text tractor, see if the results are correct.
Expected valid inputs are “car-race”, “deer”, “guitar-violin”, “gym”, “helicopter”, “carousel”, “monkeys-trees”, “golf”, “rollercoaster”, “horsejump-stick”, “planes-crossing”, “tractor”
Try ticking a search result, and ask a question in the leftside chatbox about the selected media.
Note
For each chat request, you may select either a single image, or multiple images, or a single video. Multiple videos or a collection of images+videos are not supported yet.
Performance#
You can check the end-to-end response time for each round of question-and-answering in the chat history.
Summary#
In this get started guide, you learned how to:
Build the microservice images
Deploy the application with the microservices
Try the application with a demo dataset
Learn More#
Check System requirements.
Learn how to deploy the application with Helm.
Explore more functionalities in Tutorials.
Understand the components, services, architecture, and data flow, in Overview.
Check Troubleshooting.