# Get Started The Live Video Captioning RAG sample application is a retrieval-augmented generation workflow that creates caption-text embeddings and stores them in a vector database together with the corresponding video frames and metadata, using an LLM that is optimized and deployed using OpenVINO™ toolkit, for response generation. The application works with the [Live Video Captioning](https://docs.openedgeplatform.intel.com/2026.3/edge-ai-suites/live-video-captioning/index.html) sample application that processes a Real-Time Streaming Protocol (RTSP) video stream, runs video analytics pipelines, and uses a Vision-Language Model (VLM) to generate live captions for video frames. The Live Video Captioning sample application then sends the frame data, caption text, and associated metadata to the Live Video Captioning RAG sample application so the latter can build an embedding context and store it in the vector database. The Live Video Captioning RAG sample application then provides chatbots that answer questions based on the caption text generated from the video frames. By following this guide, you will learn how to: - **Set up the sample application**: Use Docker Compose tool to deploy the application in your system environment. - **Run the sample application**: Launch the application and use the chatbots to answer questions. - **Customize application parameters**: Customize settings, for example, the LLM models and deployment configurations, to adapt the application to your specific requirements and environment. ## Prerequisites - Verify that your system meets the minimum requirements. See [System Requirements](./get-started/system-requirements.md) for details. - Install Docker platform: [Installation Guide](https://docs.docker.com/get-docker/). - Install Docker Compose tool: [Installation Guide](https://docs.docker.com/compose/install/). - OpenVINO toolkit-compatible VLM/LLM. Follow the guide in [section](#3-download-models-one-time) below. - If the sample application is used with RTSP streams, set up the RTSP stream source (live camera or test feed), or simulated RTSP stream source using local video files, separately. Refer to [the instructions](https://docs.openedgeplatform.intel.com/2026.3/edge-ai-suites/live-video-captioning/get-started/simulated-rtsp-stream-guide.html) for details. ## Run the Application ### 1. Clone the suite Go to the target directory of your choice and clone the suite. If you want to clone a specific release branch, replace `main` with the desired tag. To learn more on partial cloning, check the [Repository Cloning guide](https://docs.openedgeplatform.intel.com/2026.3/OEP-articles/contribution-guide.html#repository-cloning-partial-cloning). > [!NOTE] > `live-video-captioning-rag` builds and runs alongside its sibling sample, `live-video-captioning` > (shared `config.json`, `mosquitto.conf`, and app source used as a Docker build context), so both > directories must be included in the sparse checkout. ```bash git clone --filter=blob:none --sparse --branch release-2026.3.0 https://github.com/open-edge-platform/metro-ai-suite.git cd metro-ai-suite git sparse-checkout set live-video-analysis/live-video-captioning-rag live-video-analysis/live-video-captioning cd live-video-analysis/live-video-captioning-rag ``` ### 2. Create `.env` Run the setup helper: ```bash bash scripts/setup_env.sh ``` The helper creates `.env` from `.env.example`, detects `HOST_IP`, and stores image settings such as `REGISTRY` and `TAG` in the file. Use `--force` only if you want to overwrite an existing `.env`: ```bash bash scripts/setup_env.sh --force ``` This script sets the following important values: | Variable | Default | Purpose | | --- | --- | --- | | `HOST_IP` | `(auto-detected)` | Host address advertised to browsers on other machines for WebRTC media. Not needed when the browser runs on the same host. | | `REGISTRY` | `intel/` | Image registry prefix. | | `TAG` | `2026.3.0-rc1` | Docker image tag. | | `LVC_DASHBOARD_PORT` | `4173` | Port for the LVC web dashboard. | | `LVC_RAG_DASHBOARD_PORT` | `4172` | Port for the LVC-RAG web dashboard. | | `EVAM_HOST_PORT` | `8040` | Port for the pipeline management REST API. | | `WHIP_SERVER_PORT` | `8889` | Port for WebRTC/WHIP signaling (mediamtx). | | `MQTT_PORT` | `1883` | Port for the internal MQTT broker. | | `WEBRTC_BITRATE` | `2048` | WebRTC stream bitrate in kbps. Lower values reduce bandwidth. | | `ENABLE_DETECTION_PIPELINE` | `false` | Enables optional object-detection pre-filtering when set to `true`. | | `ALERT_MODE` | `false` | Enables alert-style visual highlighting based on keyword rules when set to `true`. | | `CAPTION_HISTORY` | `3` | Number of previous captions shown in the UI. | | `RAG_CHATBOT_MODE` | `embedded` | RAG chatbot mode: `embedded (in-dashboard)` or `detached (browser tab)` | | `DEFAULT_RTSP_URL` | *(empty)* | Pre-fills the RTSP URL field in the dashboard on load. | | `HUGGINGFACEHUB_API_TOKEN` | *(empty)* | Required for downloading gated Hugging Face models. | | `MODEL_CACHE_PATH` | `/llm_models` | Host path used for cached/downloaded model artifacts. | | `EMBEDDING_MODEL_NAME` | `QwenText/qwen3-embedding-0.6b` | Embedding model identifier used by embedding service configuration. | | `EMBEDDING_DEVICE` | `CPU` | Target device for embedding inference runtime (for example `CPU`, `GPU`, `NPU`). | | `LLM_MODEL_ID` | `microsoft/Phi-3.5-mini-instruct` | LLM model identifier used for RAG response generation. | | `LLM_DEVICE` | `CPU` | Target device for LLM inference runtime (for example `CPU`, `GPU`, or `NPU`). | | `MAX_TOKENS` | `1024` | Maximum number of generated output tokens per response. | | `TOP_K` | `1` | Number of top retrieved context entries used during RAG answering. | | `SCORE_THRESHOLD` | `0.5` | Minimum retrieval similarity score required to include context. | | `MAX_PROMPT_LEN` | `1024` | Maximum token limit accepted by the LLM pipeline on NPU devices. | | `VDMS_HOST` | `vdms-vector-db` | Hostname of the VDMS vector database service used by the app. | | `VDMS_VDB_HOST` | `vdms-vector-db` | Vector DB hostname used by compatibility paths in the backend stack. | ### 3. Download models (one-time) #### Gated Hugging Face models Some models (for example, Gemma-3) require a Hugging Face access token. Set the token in `.env` or export it before running the download script: ```bash export HUGGINGFACEHUB_API_TOKEN= ``` #### Specifying the conversion device By default the model is converted on CPU. To explicitly set the device: ```bash ./model_download_scripts/download_models.sh \ --model \ --type vlm \ --weight-format int8 \ --device ``` > [!NOTE] > NPU currently requires `int4` quantization for VLM/LLM conversion. If you pass `--device NPU` with `int8` or `fp16`, the script automatically overrides it to `int4`. #### Download a VLM model Download the VLM model used for generating captions in Live Video Captioning (LVC). The following example demonstrates deployment on a `CPU` device: ```bash ./model_download_scripts/download_models.sh \ --model OpenGVLab/InternVL2-1B \ --type vlm \ --weight-format int8 ``` The VLM models stored under `ov_models`. #### Download a LLM model Download the LLM model used for Retrieval-Augmented Generation (RAG). The following example demonstrates deployment on a `CPU` device: ```bash # Set --model to the Hugging Face model you want to convert. # Set --device to the preferred conversion target (for example, CPU or GPU). # Set --weight-format to the precision/quantization format (`int4`, `int8`, or `fp16`). ./model_download_scripts/download_models.sh \ --model microsoft/Phi-3.5-mini-instruct \ --type llm \ --device CPU \ --weight-format int8 ``` > [!NOTE] > NPU support is limited to a subset of LLMs. Refer to the [OpenVINO LLMs Hugging Face collection](https://huggingface.co/collections/OpenVINO/llms-optimized-for-npu) > for supported models. Open the model card, identify the `Original model`, and use that model > name when downloading the model with the download script. This stores the model under `llm_models/`. See [Model Preparation](https://docs.openedgeplatform.intel.com/2026.3/edge-ai-suites/live-video-captioning/get-started/model-preparation.html) for detailed usage. ### 4. Start the application From the `live-video-analysis/live-video-captioning-rag` directory, start the sample application using the Docker Compose tool: ```bash docker compose up -d ``` > [!NOTE] > Docker Compose automatically reads values from `.env` in the project root. > > --- > > The application will take some time to start. Check the container status and ensure that > their status is `"healthy/running"` by using the `docker ps` command before accessing the application. ### 5. Access the application Follow these steps to use the application: 1. Open the Live Video Captioning UI: Open the dashboard from a browser on the same host at `http://localhost:4173`. From another machine on the network, use `http://:4173` instead. 2. Start a captioning run with a valid RTSP stream. 3. Confirm that captions are being generated. 4. Click the `chat icon` in the top bar (visible only when embedding is enabled). 5. This opens the Live Caption RAG chatbot. 6. Use the chat panel to ask questions based on the generated scene captions. ### 6. Stop the application ```bash docker compose down ``` ## Troubleshooting ### Live Caption RAG dashboard Does not Open or is Unreachable - Confirm that `live-video-captioning-rag` container is running. - Confirm that port mapping `${LIVE_VIDEO_RAG_HOST_PORT:-4172}:4172` is available. - Check `http://localhost:4172/api/health`. ### Caption pipeline with RTSP not Running in LVC Dashboard - If your network uses a proxy, add your RTSP stream host or IP to `no_proxy` so the stream connection does not go through the proxy. - For more detail on LVC, please refer to the [LVC Documentation](https://docs.openedgeplatform.intel.com/2026.3/edge-ai-suites/live-video-captioning/get-started.html) ### Embeddings are Not Being Stored - Ensure that the caption pipeline is actively running (not running means no ingestion). - Verify the embedding service health on `http://localhost:9777/health`. - Verify that the VDMS container is running. - If containers are running but no embeddings are stored, remove the volume and restart the services: ```bash docker volume rm live-video-caption_vdms-db ``` ## Advanced paths - [Build from Source](./get-started/build-from-source.md) - [Deploy with Helm](./get-started/deploy-with-helm.md) - [Run Unit Tests](./get-started/run-unit-tests.md) - [API Reference](./api-reference.md) - [Known Issues](./known-issues.md) :::{toctree} :hidden: ./get-started/system-requirements.md ./get-started/build-from-source.md ./get-started/deploy-with-helm.md ./get-started/run-unit-tests.md :::