Quick Start: Live Video Captioning RAG#

Get the application up and running in a few simple steps.

Live Video Captioning RAG works with Live Video Captioning to generate live captions from a video stream, store caption context in a vector database, and let you ask natural-language questions about what happened in the video.

Note:

  1. Setup and first run time depends on network speed because Docker images and models are downloaded.

  2. If you do not have a camera stream, configure a simulated RTSP stream by following these instructions.

Before You Begin#

Make sure your machine meets these minimums:

What

Minimum

Processor

Intel(R) Core(TM) Ultra (2nd or 3rd gen) with integrated GPU

Memory

Min 16 GB RAM

Disk

64 GB free SSD space

OS

Ubuntu 24.04 or 24.10

Internet

Required for first-time setup

You also need Docker installed. If you do not have it yet, run the following two commands in a terminal:

curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER

After running those commands, log out and log back in for the changes to take effect.

Step 1 - Get the Code#

Open a terminal and run:

git clone --filter=blob:none --sparse --branch main https://github.com/open-edge-platform/edge-ai-suites.git
cd edge-ai-suites
git sparse-checkout set metro-ai-suite
cd metro-ai-suite/live-video-analysis/live-video-captioning-rag

Step 2 - Set Up Configuration#

Run the setup script to create the .env file and auto-detect your host IP:

bash scripts/setup_env.sh

Step 3 - Download AI Models (one-time)#

You need two models:

  1. A VLM model for Live Video Captioning.

  2. An LLM model for RAG question-answering.

Run:

# VLM model for caption generation
./model_download_scripts/download_models.sh \
	--model OpenGVLab/InternVL2-1B \
	--type vlm \
	--weight-format int8

# LLM model for RAG responses
./model_download_scripts/download_models.sh \
	--model Qwen/Qwen2.5-3B-Instruct \
	--type llm \
	--device CPU \
	--weight-format int8

If a model is gated on Hugging Face, set your token first:

export HUGGINGFACEHUB_API_TOKEN=<your-token>

Specifying the conversion device#

By default, conversion runs on CPU. To target another device:

./model_download_scripts/download_models.sh \
	--model <model_name> \
	--type <vlm|llm> \
	--weight-format int8 \
	--device <CPU|GPU|NPU>

Note: NPU support currently only works with int4 quantization when converting VLM models. If --device NPU is specified alongside int8 or fp16, the script will automatically switch the quantization to int4. Additionally, LLM model support for NPU is not yet available in Live-Video-Captioning-RAG.

Step 4 - Start the Application#

docker compose up -d

The first run may take a few minutes while images are downloaded and services become healthy.

Step 5 - Open the Dashboards#

  1. Open Live Video Captioning at:

    http://<YOUR_IP>:4173

  2. In the Live Video Captioning dashboard:

    • Enter a video source (RTSP URL or USB/webcam if available).

    • Select the VLM device and model.

    • Click Start and confirm captions are appearing.

  3. Open Live Video Captioning RAG via chat icon on the top right of Live Video Captioning dashboard or open a new tab with:

    http://<YOUR_IP>:4172

  4. Ask questions about the live or recent scene in the chat panel.

Step 6 - Stop the Application#

When you are done:

docker compose down

Troubleshooting#

Problem

What to try

RAG dashboard does not load

Wait 30-60 seconds after docker compose up -d, then refresh http://<YOUR_IP>:4172

Live captioning dashboard does not load

Verify http://<YOUR_IP>:4173 is reachable and containers are running (docker ps)

No captions appear

Check that your RTSP URL is reachable from this machine

No RAG answers/context

Ensure captioning is running first so embeddings can be ingested

Stream behind a proxy

Add the camera IP/host to no_proxy in your shell environment

“permission denied” with Docker

Run sudo usermod -aG docker $USER, then log out and back in

Model download fails with authentication error

Set HUGGINGFACEHUB_API_TOKEN and rerun the command

Model download interrupted or fails due to network

Remove model folders (ov_models/, llm_models/, ovms_model/) and rerun download

Next Steps#

After quick start setup, continue with: