Get Started#

Overview#

The VMS Adapter Plugin (VAP) bridges Video Management Systems (VMS), such as Nx Witness, Genetec, and Milestone VMS platforms, with AI Analytics Applications, such as Live Video Captioning (LVC), DL Streamer vision analytics applications like Loitering Detection. It also provides a unified provider dashboard, built with the React library, for managing cameras and analytics runs. This guide shows how to deploy the full stack with the Docker Compose tool and run your first analytics session.

This guide shows how to:

  • Set up prerequisites: Start LVC or DL Streamer vision analytics (Loitering Detection) before VAP, since VAP fetches their schemas at startup.

  • Configure the environment: Point VAP at your VMS and Analytics App services.

  • Run the provider dashboard (Optional): Discover cameras, enable streams, and start analytics runs.

Quick Start#

Check the folder layout to familiarize yourself with the code structure.

Prerequisites#

  • Verify that your system meets the minimum requirements.

  • Install the Docker platform: Installation Guide.

  • Install the Docker Compose tool: Installation Guide.

  • Nx Witness VMS with an accessible REST API (NX_HOST, NX_USERNAME, NX_PASSWORD). This document does not cover Nx Witness or Nx Cloud setup.

  • At least one Analytics Application running before VAP starts:

    • Live Video Captioning (LVC) — for Vision-Language Model (VLM) based AI captioning.

    • Loitering Detection (DL Streamer Vision based) — for real-time detection of loitering behavior in transportation hubs with Nx write-back.

Step 1 — Start Live Video Captioning (LVC)#

Note: Skip this step if you are only using Loitering Detection.

Clone and start the LVC application from its own directory. LVC must be running before VAP starts, because VAP fetches the LVC OpenAPI schema at startup to build the analytics configuration form:

git clone --filter=blob:none --sparse --branch release-2026.2.0 https://github.com/open-edge-platform/edge-ai-suites.git
cd edge-ai-suites
git sparse-checkout set metro-ai-suite
cd metro-ai-suite/live-video-analysis/live-video-captioning

Follow the LVC Get Started guide to prepare models and configure the environment, then start the stack:

docker compose up -d

Verify LVC is reachable:

curl http://localhost:4173/health

Step 2 — Start Loitering Detection#

Note: Skip this step if you are only using Live Video Captioning.

Loitering Detection is a user-provided application based on the DL Streamer Pipeline Server. Follow the Loitering Detection Get Started guide to bring up the application. Ensure all containers are running, but do not start the pipelines yet. The following services must be reachable from the VAP backend container:

Service

Default Port

Purpose

DL Streamer Pipeline Server

8080

Receive pipeline start/stop commands

MQTT Broker

1883

Publish inference metadata to VAP

Verify the DL Streamer Pipeline Server is reachable:

curl -k https://<LOITERING_DETECTION_HOST>:443/pipelines

Step 3 — Clone VAP and Create the .env File#

cd metro-ai-suite/vms-adapter-plugin
cp .env.example .env

Open .env and update the variables for your environment. Variables are grouped by scope. Within each analytics app group, Mandatory applies only if you are using that app.

Common (VAP with Nx Witness):

Variable

Description

Required?

NX_HOST / NX_USERNAME / NX_PASSWORD

Nx Witness host and credentials

Mandatory

NX_TLS_VERIFY / NX_CA_BUNDLE

Nx TLS verification toggle and optional CA bundle path (default: false)

Optional

PG_PASSWORD

PostgreSQL password (change from default)

Optional

UI_HTTPS_PORT

Host port for the dashboard HTTPS (default: 3443)

Optional

Live Video Captioning (LVC):

Variable

Description

Required?

LVC_HOST / LVC_BASE_URL

LVC backend host and base URL, e.g., http://<lvc-host>:4173

Mandatory

MEDIAMTX_URL

URL of the MediaMTX WebRTC server, e.g., http://<lvc-host>:8889

Mandatory

MQTT_BROKER_HOST / MQTT_BROKER_PORT

LVC MQTT broker host and port that VAP subscribes to for captions (default port: 1883)

Mandatory

MQTT_BROKER_TLS_ENABLED / MQTT_BROKER_CA_BUNDLE / MQTT_BROKER_CLIENT_CERT / MQTT_BROKER_CLIENT_KEY

MQTT TLS, CA bundle, and optional mutual TLS client certificate for the LVC broker subscriber

Optional

DL Streamer Vision (dls_vision — Loitering Detection):

Variable

Description

Required?

DLS_VISION_HOST / DLS_VISION_PORT

DL Streamer Pipeline Server host and port for Loitering Detection app (default port: 443)

Mandatory

DLS_PIPELINE_CPU / DLS_PIPELINE_GPU / DLS_PIPELINE_NPU

Device-specific DL Streamer pipeline names; at least one must be set

Mandatory

MQTT_HOST / MQTT_PORT

MQTT broker host and port for dls_vision metadata (default: 1883)

Mandatory

DLS_VISION_TLS_VERIFY / DLS_VISION_CA_BUNDLE

DL Streamer TLS verification toggle and optional CA bundle path (default: false)

Optional

MQTT_TLS_ENABLED / MQTT_CA_BUNDLE / MQTT_CLIENT_CERT / MQTT_CLIENT_KEY

MQTT TLS, CA bundle, and optional mutual TLS client certificate for the dls_vision subscriber

Optional

Note: If Live Video Captioning or Loitering Detection is running on the same host as VAP, use host.docker.internal (Linux/Mac). Otherwise, use the actual IP address.

For certificate path examples and TLS behavior details, see TLS and Certificate Configuration.

Step 4 — Build and Start VAP#

docker compose up -d --build

Wait for all services to become healthy:

docker compose ps

Expected output — all services should show healthy or running:

NAME                          STATUS
vms-adapter-backend           Up (healthy)
vms-adapter-ui                Up
vms-adapter-postgres          Up (healthy)

Verify the backend is up:

curl -k https://localhost:3443/v1/health

Step 5 - Run analytics from Nx Witness UI#

The steps above creates an ‘Nx REST Integration’. Applications configured with VMS Adapter Plugin are registered as part of Nx Integration and are available in Nx Client UI across cameras with analytics ready to be trigerred by the user.

  1. In the Nx Witness desktop client, close any open camera visualizer window.

  2. Navigate to the left panel, and under the server, find the camera you wish to run analytics on and right-click to open context menu.

  3. Select Camera Settings.

  4. Go to the Integrations tab.

  5. Click VAP Analytics Integration to expand the per-camera settings.

  6. The available options are present as per the application configured. Check the application check-box, select or fill the application configurations as desired, and click Apply to run the analytics.

To stop the same, unselect the application and click Apply

Stop the Stack#

docker compose down          # stop without removing data
docker compose down -v       # stop and remove PostgreSQL volume

Folder Layout#

vms-adapter/
├── plugin/                         # Backend Python package
│   ├── base/
│   │   └── interfaces.py           #  IVmsShim + IAnalyticsAppShim abstract interfaces
│   ├── common/
│   │   └── schema_builder.py       #  Dynamic Pydantic model builder from JSON Schema
│   └── Analytics/
│       ├── api/
│       │   ├── routes/
│       │   │   ├── cameras.py      #   Camera discovery + enable/disable
│       │   │   ├── analytics_apps.py    #   Generic Analytics App API (discover, runs, stream, options)
│       │   │   ├── events.py       #   Event timeline
│       │   │   ├── analysis.py     #   Analysis result callback
│       │   │   ├── sessions.py     #   Session tracking
│       │   │   ├── vms.py          #   VMS register
│       │   │   ├── health.py       #   Health + readiness
│       │   │   └── config.py       #   Config status
│       │   └── deps.py             #   FastAPI dependency injection
│       ├── db/
│       │   └── repository.py       #   Async SQLAlchemy CRUD
│       ├── models/
│       │   ├── db.py               #   ORM models (Camera, Event, Session, …)
│       │   └── domain.py           #   Domain dataclasses
│       ├── pipeline/
│       │   └── orchestrator.py     #   Background camera sync + event processing
│       ├── config.py               #   Pydantic settings (YAML + env)
│       ├── factory.py              #   Shim factory
│       └── main.py                 #   FastAPI application entry point
│
├── vms_shim/                       # Concrete VMS shims
│   ├── frigate/
│   │   ├── shim.py                 #  FrigateVmsShim — discovers cameras via local config
│   │   └── config/                 #  Frigate config.yml (cameras, go2rtc, etc.)
│   └── nxwitness/
│       └── shim.py                 #  NxWitnessVmsShim — Nx Witness REST API v4
│
├── analytics_app_shim/                  # Concrete Analytics App shims
│   └── lvc/
│       ├── api_client.py           #  LvcApiClient — all HTTP calls to LVC backend
│       ├── schema.py               #  LvcSchemaManager — OpenAPI fetch, $ref resolution,
│       │                           #    UI annotations, Pydantic model building
│       └── shim.py                 #  LiveCaptioningAnalyticsAppShim — composes api_client + schema
│
├── ui/                             # React 19 / Vite frontend served by nginx
│   ├── src/
│   │   ├── App.jsx                 #  Root component + state
│   │   ├── components/MainPage/
│   │   │   ├── CameraDiscoveryPanel.jsx
│   │   │   ├── AnalyticsEnginePanel.jsx   # Dynamic schema form + run lifecycle
│   │   │   ├── SchemaForm.jsx             # Generic JSON Schema → form renderer
│   │   │   ├── LiveStreamTab.jsx          # WebRTC player + caption overlay
│   │   │   └── AnalysisResultsPanel.jsx
│   │   ├── hooks/
│   │   │   └── useLvcStream.js     #  SSE caption stream hook
│   │   └── services/
│   │       └── api.js              #  Generic API client functions
│   └── nginx.conf                  #  Reverse proxy: /v1 → backend, /whep → MediaMTX
│
├── config/
│   └── config.yaml                 # Runtime config (cameras, VMS endpoints, LVC URL)
├── tests/                          # pytest unit + integration tests
├── Dockerfile                      # Backend image
├── docker-compose.yml              # backend + ui + postgres
├── pyproject.toml                  # Python deps + package config
└── .env.example                    # Environment variable reference

Next Steps#

  1. Explore the Architecture: Learn how VAP components interact in the How It Works guide.

  2. Follow Integration Tutorials: Use the How-To Guides for end-to-end walkthroughs of LVC and Loitering Detection integrations.

  3. Browse the API: Explore all available endpoints in the API Reference.

  4. Troubleshooting: If you encounter issues, check the Troubleshooting Guide.