# Get Started Guide - **Time to Complete:** 30 mins - **Programming Language:** Python ## Get Started ### Prerequisites - Install Docker: [Installation Guide](https://docs.docker.com/get-docker/). - Install Docker Compose: [Installation Guide](https://docs.docker.com/compose/install/). - Install Intel Client GPU driver: [Installation Guide](https://dgpu-docs.intel.com/driver/client/overview.html). ### Step 1: Get the docker images #### Option 1: Build from source Go to the target directory of your choice and clone the suite. If you want to clone a specific release branch, replace `main` with the desired tag. To learn more on partial cloning, check the [Repository Cloning guide](https://docs.openedgeplatform.intel.com/2026.3/OEP-articles/contribution-guide.html#repository-cloning-partial-cloning). ```bash git clone --filter=blob:none --branch main https://github.com/open-edge-platform/metro-ai-suite.git cd metro-ai-suite ``` Run the commands to build images for the microservices: ```bash git clone https://github.com/open-edge-platform/edge-ai-libraries.git -b main cd edge-ai-libraries/microservices cd visual-data-preparation-for-retrieval/multimodal-dataprep ./build.sh cd ../../ docker build -t vector-retriever-milvus:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg no_proxy=$no_proxy -f vector-retriever/milvus/src/Dockerfile . cd multimodal-embedding-serving docker build -t multimodal-embedding-serving:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg no_proxy=$no_proxy -f docker/Dockerfile . cd ../../.. ``` VLM OpenVINO Serving is not built from source. The Compose file references the published image by its pinned tag, so Compose pulls it automatically. To fetch it ahead of time: ```bash docker pull intel/vlm-openvino-serving:2026.2.0 ``` Run the command to build image for the application: ```bash docker build -t visual-search-qa-app:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy --build-arg no_proxy=$no_proxy -f visual-search-question-and-answering/src/Dockerfile . ``` #### Option 2: use remote prebuilt images Set a remote registry by exporting environment variables: ```bash export REGISTRY="intel/" export TAG="latest" ``` ### Step 2: Prepare host directories for models and data ```bash mkdir -p $HOME/data ``` If you would like to test the application with a demo dataset, please continue and follow the instructions in the [Try with a demo dataset](#try-with-a-demo-dataset) section later in this guide. Otherwise, if you would like to use your own data (images and video), put it all in a directory on the host and point `HOST_DATA_PATH` at it before deploying the services, for example `export HOST_DATA_PATH="$HOME/data"`. It defaults to `$HOME/data`. > [!NOTE] > Supported media types are jpg, png, and mp4. ### Step 3: Deploy #### Option1 (**Recommended**): Deploy with docker compose 1. Go to the deployment files ``` bash cd visual-search-question-and-answering/ cd deployment/docker-compose/ ``` 2. Set up environment variables. > [!NOTE] > You need to set models first. - **Ubuntu**: ``` bash export EMBEDDING_MODEL_NAME="CLIP/clip-vit-h-14" # Replace with other models if needed export VLM_MODEL_NAME="Qwen/Qwen2.5-VL-7B-Instruct" # Replace with other models if needed source env.sh ``` > [!IMPORTANT] > You must set `EMBEDDING_MODEL_NAME` and `VLM_MODEL_NAME` before running `env.sh`. See > [Supported models](https://docs.openedgeplatform.intel.com/2026.3/edge-ai-libraries/multimodal-embedding-serving/supported-models.html) for Multimodal Embedding Serving for available embedding models, and > [Supported models](https://github.com/open-edge-platform/edge-ai-libraries/blob/release-2026.2.0/microservices/vlm-openvino-serving/docs/user-guide/Overview.md#models-supported) for VLM OpenVINO for available VLM models. `env.sh` only sets a variable when it is not already exported, so anything you export beforehand wins. Besides the two model names, the settings most often changed are: | Variable | Purpose | Default | | --- | --- | --- | | `HOST_DATA_PATH` | Host directory holding the media to index, mounted into the services | `$HOME/data` | | `HF_ENDPOINT` | Hugging Face endpoint used to download models | `https://hf-mirror.com` | | `VLM_COMPRESSION_WEIGHT_FORMAT` | VLM weight precision, for example `int4` to reduce memory | `int8` | > [!IMPORTANT] > `HF_ENDPOINT` defaults to a mirror intended for mainland China. Set > `export HF_ENDPOINT="https://huggingface.co"` to download models from > Hugging Face directly. Each microservice picks its inference device independently, so that bulk ingestion and interactive queries do not have to share one accelerator: | Variable | Microservice | Default | | --- | --- | --- | | `VLM_DEVICE` | `vlm-openvino-serving`, question answering | `GPU` | | `EMBEDDING_DEVICE` | `multimodal-embedding-serving`, query embedding | `CPU` | | `DATAPREP_EMBEDDING_DEVICE` | `multimodal-dataprep`, ingest embedding | `GPU` | | `DATAPREP_DETECTION_DEVICE` | `multimodal-dataprep`, object detection | `GPU` | Each accepts `CPU`, `GPU` or `NPU`. The ingest pipeline is the throughput bottleneck, so it takes the GPU by default, while query embedding is a single short request per search and runs on the CPU. Avoid placing the embedding model on the same GPU for both services unless it has memory to spare: two copies of a large model, such as `CLIP/clip-vit-h-14`, can exhaust the device and the next query then fails with `CL_OUT_OF_RESOURCES`. `GPU` selects the first GPU that OpenVINO reports. On a platform with an integrated GPU as `GPU.0` and a discrete GPU as `GPU.1`, name the one you want explicitly. Refer to [OpenVINO's query device sample](https://docs.openvino.ai/2024/learn-openvino/openvino-samples/hello-query-device.html) to list the available indices. For example, to run everything on the CPU: ``` bash export VLM_DEVICE=CPU export EMBEDDING_DEVICE=CPU export DATAPREP_EMBEDDING_DEVICE=CPU export DATAPREP_DETECTION_DEVICE=CPU ``` > [!NOTE] > `MM_DATAPREP_EMBEDDING_DEVICE` and `MM_DATAPREP_DETECTION_DEVICE` are the > previous names of the two dataprep variables and are still honoured, so > existing scripts keep working. `DATAPREP_*` takes precedence if both are > set. Milvus/MinIO/etcd state now uses Docker named volumes by default. For cleanup/reset, use `source env.sh --clean-data` (this preserves `ov-models` and `dataprep-detection-models`). Object detection is disabled by default, so only full frames are indexed. To also embed detected object crops, set `DATA_INGEST_WITH_DETECT=true` and `DATAPREP_DETECTION_MODEL` before running `env.sh`. `DATAPREP_DETECTION_MODEL` takes a Hugging Face OpenVINO object-detection model in `OpenVINO/` form, for example `OpenVINO/yolox_s-int8-ov`. It has no default, so ingestion fails if detection is enabled without it. The model is downloaded on first use and cached in the `dataprep-detection-models` volume; licenses differ per model, so review the model card before enabling it. Optionally set `DATAPREP_DETECTION_CLASSES` to a comma-separated allowlist such as `person,car` to keep only those classes. - **EMT-S**: If you are on an EMT-S platform, set up the variables correspondingly by running: ``` bash cd emt-s # go to emt-s specific files export EMBEDDING_MODEL_NAME="CLIP/clip-vit-h-14" # Replace with other models if needed export VLM_MODEL_NAME="Qwen/Qwen2.5-VL-7B-Instruct" # Replace with other models if needed source env.sh ``` 3. Deploy with docker compose ``` bash docker compose -f compose_milvus.yaml up -d ``` It might take a while to start the services for the first time, as there are some models to be prepared. Check if all microservices are up and runnning with ```sh docker compose -f compose_milvus.yaml ps ``` Output: ```text NAME COMMAND SERVICE STATUS PORTS multimodal-dataprep "uvicorn src.main:ap…" multimodal-dataprep running (healthy) 0.0.0.0:9990->8000/tcp, :::9990->8000/tcp milvus-etcd "etcd -advertise-cli…" milvus-etcd running (healthy) 2379-2380/tcp milvus-minio "/usr/bin/docker-ent…" milvus-minio running (healthy) 0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp milvus-standalone "/tini -- milvus run…" milvus-standalone running (healthy) 0.0.0.0:9091->9091/tcp, 0.0.0.0:19530->19530/tcp, :::9091->9091/tcp, :::19530->19530/tcp multimodal-embedding gunicorn -b 0.0.0.0:8000 - ... Up (unhealthy) 0.0.0.0:9777->8000/tcp,:::9777->8000/tcp retriever-milvus "uvicorn retriever_s…" retriever-milvus running (healthy) 0.0.0.0:7770->7770/tcp, :::7770->7770/tcp visual-search-qa-app "streamlit run app.p…" visual-search-qa-app running (healthy) 0.0.0.0:17580->17580/tcp, :::17580->17580/tcp vlm-openvino-serving "/bin/bash -c '/app/…" vlm-openvino-serving running (healthy) 0.0.0.0:9764->8000/tcp, :::9764->8000/tcp ``` #### Option2: Deploy in Kubernetes Refer to [Deploy with helm](./get-started/deploy-with-helm.md) for details. ## Try with a demo dataset *Applicable to deployment with Option 1 (docker compose deployment). ### Prepare demo dataset [DAVIS](https://davischallenge.org/davis2017/code.html) Use the `prepare_demo_dataset.sh` script shipped in `tests/`. It runs on the host (it needs `curl`, `unzip` and `ffmpeg`) and writes the dataset into the directory shared with the dataprep microservice: ```bash bash tests/prepare_demo_dataset.sh ``` Check your host data directory `$HOME/data` (or `$HOST_DATA_PATH`) and see if `DAVIS` is there. To save time, only a subset of the dataset is prepared: it is stored in `$HOME/data/DAVIS/subset`, half as videos and half as sampled images. Use this path in the next step. The script also writes a `meta/.json` sidecar per item with a `camera` and a `capture_date` field, so the `camera` and date filters in the UI have values to filter on. ### Use it on Web UI Go to `http://{host_ip}:17580` with a browser. Put the exact path to the subset of demo dataset (usually`/home/user/data/DAVIS/subset`, may vary according to your local username) into `file directory on host`. Click `UpdataDB` and wait for the uploading done. Try searching with query text `tractor`, see if the results are correct. Expected valid inputs are "car-race", "deer", "guitar-violin", "gym", "helicopter", "carousel", "monkeys-trees", "golf", "rollercoaster", "horsejump-stick", "planes-crossing", "tractor" Try ticking a search result, and ask a question in the leftside chatbox about the selected media. > [!NOTE] > For each chat request, you may select either a single image, or multiple images, or a single video. Multiple videos or a collection of images+videos are not supported yet. ## Performance You can check the end-to-end response time for each round of question-and-answering in the chat history. ## Summary In this get started guide, you learned how to: - Build the microservice images - Deploy the application with the microservices - Try the application with a demo dataset ## Learn More - Check [System requirements](./get-started/system-requirements.md). - Learn how to deploy the application with [Helm](./get-started/deploy-with-helm.md). - Explore more functionalities in [Tutorials](./tutorials.md). - Understand the components, services, architecture, and data flow, in [Overview](./index.md). - Check [Troubleshooting](./troubleshooting.md). :::{toctree} :hidden: ./get-started/system-requirements ./get-started/deploy-with-helm :::