# Get Started (UAV Mission Compute SDK Mode)
This guide provides a step-by-step walkthrough for testing the UAV Vision Analytics application in UAV Mission Compute SDK mode and running the demo with a simulated UAV camera feed/RealSense cameras.
## How It Works
A minimal stack. Telemetry is received via MQTT from the `uav-mission-compute-sdk` project, which must be started first. The DL Streamer Pipeline Server container reads armed/disarmed state from `uav/{id}/telemetry/status` and subscribes to three RTSP camera streams (nadir, forward, rear).

**Telemetry / pipeline lifecycle flow:**
```mermaid
sequenceDiagram
participant Client as curl / QGroundControl / ffplay
participant Nginx as nginx (TLS :443, RTSP :8555)
participant DL_Streamer_Pipeline_Server as DL Streamer Pipeline Server
participant SDK as uav-mission-compute-sdk
participant OVL as gvapython (MavlinkReceiver)
participant Frame as Video Frame
Client->>Nginx: POST /pipelines/... (HTTPS :443)
Nginx->>DL_Streamer_Pipeline_Server: proxy_pass REST :8081
DL_Streamer_Pipeline_Server-->>Nginx: 200 instance_id
Nginx-->>Client: 200 instance_id
SDK->>OVL: broadcast MQTT Telemetry :1883
Note over OVL: background thread parses
GLOBAL_POSITION_INT, VFR_HUD,
GPS_RAW_INT into latest_data
Frame->>OVL: process_frame() per frame
OVL->>Frame: ROI labels (ALT · SPD · HDG · LAT · LON · SATS)
Frame->>Nginx: annotated RTSP :8555 (stream {} passthrough)
Nginx->>Client: rtsp://:8555/...
```
**Services:**
| Service | Image | Ports | Role |
| --- | --- | --- | --- |
| `dlstreamer-pipeline-server` | `intel/dlstreamer-pipeline-server` | `8081`, `8555` | AI inference, RTSP output |
| `metrics-manager` (container `uav-va-metrics-manager`) | `intel/metrics-manager` | `9090` (internal; through nginx at `/metrics/stream` and `/api/v1/metrics/latest`) | CPU, GPU, NPU, and memory metrics for the ground-station dashboard (the container is not named `metrics-manager`, which the SDK's observability profile uses) |
| `nginx` | `nginx:1.27-alpine` | `80` (redirects to 443), `443` (HTTPS, self-signed cert), `8555` (RTSP passthrough) — published to `HOST_IP` | Reverse proxy / TLS termination — the only service that publishes ports to the host |
> [!CAUTION]
> `dlstreamer-pipeline-server` no longer publishes ports directly — it is reachable only
through `nginx` on the internal `app_network`.
---
## Steps to Test the Application
### Prerequisites
- Docker and Docker Compose v2
- Intel platform with at least 16 GB RAM (Panther Lake recommended)
- Network access to pull Docker images (configure proxy if behind a corporate firewall)
- The following system packages:
```bash
sudo apt install -y python3.12-venv ffmpeg
```
> [!NOTE]
> `python3.12-venv` is required by `make model` to create a Python virtual environment.
> `ffmpeg` provides `ffplay` for viewing the RTSP output stream and `ffmpeg` for recording.
### 1. Start the UAV Mission Compute SDK
There are two options available to get the application source:
::::{tab-set}
:::{tab-item} **Download the ZIP (recommended)**
:sync: zip
Download the compressed file and get into the directory:
```bash
curl -OjL https://github.com/open-edge-platform/federal-and-aerospace-ai-suite/releases/download/fedaero-latest/uav-mission-apps.zip
```
Decompress the downloaded file:
```bash
unzip uav-mission-apps.zip
cd uav-mission-compute-sdk/
```
:::
:::{tab-item} **Clone the repository**
:sync: clone
Clone the repository, get into the directory and start the SDK's core infrastructure (PX4, MQTT broker, MediaMTX RTSP server).
```bash
git clone https://github.com/open-edge-platform/federal-and-aerospace-ai-suite.git --branch main
cd federal-and-aerospace-ai-suite/uav-mission-compute-sdk
```
:::
::::
Then, for either option, initialize the environment:
```bash
make init # create .env, detect GPU
```
> [!CAUTION]
> Follow only **Step 0** (configure credentials) and **Step 1+2** (`make up-sim-camera`) from the [get-started guide](https://github.com/open-edge-platform/federal-and-aerospace-ai-suite/blob/release-2026.3.0/uav-mission-compute-sdk/docs/user-guide/get-started.md) / [SDK README](https://github.com/open-edge-platform/federal-and-aerospace-ai-suite/blob/release-2026.3.0/uav-mission-compute-sdk/README.md). Do **not** run `make apps` (SDK Step 3) — that starts the SDK's own AI vision-processor and dashboard, which is not needed here since `uav-vision-analytics` runs its own inference via DL Streamer Pipeline Server.
The SDK's `.env` defaults to `HOST_IP=127.0.0.1`, which binds MQTT, RTSP, and all other published ports to loopback only. Since `uav-vision-analytics` runs in a separate Docker container/network, it cannot reach loopback-bound ports. Set the SDK's `.env` to bind on all interfaces before starting it:
```bash
sed -i 's|^HOST_IP=.*|HOST_IP=0.0.0.0|' .env
make up-sim-camera # start PX4, MQTT, RTSP server
```
### 2. Configure environment
From the `uav-mission-compute-sdk` directory, get into the application directory (it sits next to the SDK in both the ZIP and the cloned repository):
```bash
cd ../uav-vision-analytics
```
Then initialize the environment:
```bash
make init
```
`make init` creates `.env` from the template and auto-detects your **host IP** (`HOST_IP`), your **Intel GPU device paths** (`GPU_DEVICE`, `GPU_RENDER_DEVICE`), **Intel NPU** (`NPU_DEVICE`), and **RealSense / USB camera** (`CAMERA_DEVICE_1`). It skips if `.env` already exists. Open and update `.env` only if `HOST_IP` is not the address other machines use to reach this host (for example on a host with several network cards).
### 3. Prepare the models
Download and export the YOLO11s detection model to OpenVINO™ FP16 IR:
```bash
make model
```
If any camera in `configs/cameras-uavsdk.yaml` has `captioning.enabled: true`, also download and convert the VLM that writes the scene captions (InternVL2-1B, OpenVINO™ IR for the CPU and the GPU):
```bash
make vlm-model
```
Both targets show the model license and ask for confirmation; add `ACCEPT_LICENSE=y` to accept without the prompt.
> [!TIP]
> See the [AI Model guide](../how-to-customize/model-and-video-management/model.md) and the
[VLM Model guide](../how-to-customize/model-and-video-management/vlm-model.md) for model details.
### 4. Start the uav-vision-analytics application
The SDK's core infrastructure (step 1) must already be running.
```bash
make uavsdk-up
```
### 5. Run a simple mission
> [!NOTE]
> Video streams are not available until the UAV is armed and actively on a mission.
Run the simple UAV mission in a persistent terminal window to keep the simulation active. The following sequence arms the UAV, commands a takeoff to 10 m, holds for 120 seconds, then lands:
```bash
curl -X POST http://localhost:8080/action/arm
curl -sf -X POST http://localhost:8080/action/takeoff \
-H "Content-Type: application/json" \
-d '{"altitude": 10}'
sleep 120
curl -X POST http://localhost:8080/action/land
```
### 6. Start inference pipelines
> [!NOTE]
> Open a new terminal window and launch the inference pipeline to begin processing the video stream.
Two options are available depending on your use case:
#### Option A — Managed RTSP output (recommended)
Runs `pipeline_manager.py` inside the DL Streamer Pipeline Server container. It watches the SDK's MQTT status topic `uav//telemetry/status` (`{"armed": true|false}`), starts the cameras of [`configs/cameras-uavsdk.yaml`](https://github.com/open-edge-platform/federal-and-aerospace-ai-suite/blob/release-2026.3.0/uav-vision-analytics/configs/cameras-uavsdk.yaml) when the drone arms and stops them when it disarms. Annotated frames are served as RTSP on port `8555`.
The cameras are RTSP streams from the SDK (`rtsp://host.docker.internal:8554//`, `UAV_ID` from `.env`, default `uav-1`). By default, `nadir_camera` detects on the CPU, `forward_camera` on the GPU, and `rear_camera` on the NPU (skipped when the host has no NPU). A camera whose RTSP source is not available is skipped. Edit the file to change the devices, switch off VLM captions, or hide the overlay; it is read again at every arming. It has the same format as for the simulator, see [Configure the cameras](../how-to-customize/camera-management/configure-cameras.md).
```bash
make check-cameras # validate cameras-uavsdk.yaml and show what would run
make start-rtsp # wait for the drone to arm, then start the cameras
make cameras-status # compare the file with what is running
```
**Captions.** With `captioning.enabled: true` on a camera, the captions are published to MQTT topic `uav/caption/` on the
**SDK's broker** (`mqtt-broker`, host port `1883`; `MQTT_BROKER_PORT` in `.env` overrides it), not on a broker of this stack: the compose file points the pipelines'
`MQTT_HOST`/`MQTT_PORT` at `host.docker.internal:1883`. Watch them with
`docker exec mqtt-broker mosquitto_sub -t 'uav/caption/#'`. The ground-station dashboard's `VIZ_REMOTE_MQTT_PORT` (default `1883`) already matches.
Stop the standalone stack first: it publishes the same port `1883` on `HOST_IP`.
The streams are available only after the drone arms — see [Step 7 — View the output stream](#7-view-the-output-stream) for the RTSP URLs.
#### Option B — Manual REST API
> [!NOTE]
> RTSP streams are not available until the UAV is armed. Run a simple mission first (see [Step 5: Run a simple mission](#5-run-a-simple-mission)).
Start a single camera pipeline directly. The UAVSDK mode loads `config-uavsdk.json`, which has the same pipeline names as the simulator stack (`uav_object_detection_` and `uav_object_detection__`) with an RTSP source, taken from the `rtsp_url` parameter.
```bash
# Detection on the CPU for the nadir camera
INSTANCE_ID=$(curl -k -s -X POST \
https:///pipelines/user_defined_pipelines/uav_object_detection_cpu \
-H "Content-Type: application/json" \
-d '{
"destination": {"frame": {"type": "rtsp", "path": "uav-nadir-camera", "encoding": "h264"}},
"parameters": {
"rtsp_url": "rtsp://host.docker.internal:8554/uav-1/nadir",
"detection_instance_id": "instcpu_yolo11s",
"overlay_label": "nadir-CPU"
}
}' | tr -d '"')
echo "Instance ID: $INSTANCE_ID"
# Verify it reached RUNNING state (not ERROR)
curl -k -s https:///pipelines/${INSTANCE_ID}/status | python3 -m json.tool
```
If `state` is `ERROR`, check the container logs:
```bash
docker logs uav-va-dlstreamer-pipeline-server 2>&1 | tail -20
```
To use another device change the pipeline name (`uav_object_detection_gpu`, `uav_object_detection_npu`, or the `_` variants with `captioner_model_path` and `mqtt_topic` in `parameters`) and the `detection_instance_id`.
> [!NOTE]
> `https:///...` is the DL Streamer Pipeline Server REST API, proxied by nginx on
> port `443` (self-signed cert; use `curl -k` to skip verification). Plain HTTP on port `80`
> redirects to HTTPS. Replace `` with the value auto-detected by `make init` (see `.env`).
### 7. View the output stream
#### View with ffplay
Install ffmpeg first if not present using `sudo apt install ffmpeg`.
Any of the annotated streams can be viewed with `ffplay -rtsp_transport tcp `.
The stream is served over RTSP/TCP only (port 8555 is the only port published). Without
`-rtsp_transport tcp`, ffplay negotiates UDP and the window stays blank.
```bash
ffplay -rtsp_transport tcp rtsp://:8555/uav-nadir-camera # nadir camera
ffplay -rtsp_transport tcp rtsp://:8555/uav-forward-camera # forward camera
ffplay -rtsp_transport tcp rtsp://:8555/uav-rear-camera # rear camera
```
#### Capture all the video streams
Record all three streams to disk with `ffmpeg`:
```bash
ffmpeg \
-rtsp_transport tcp -i rtsp://:8555/uav-nadir-camera \
-rtsp_transport tcp -i rtsp://:8555/uav-forward-camera \
-rtsp_transport tcp -i rtsp://:8555/uav-rear-camera \
-map 0:v -c:v copy nadir.mkv \
-map 1:v -c:v copy forward.mkv \
-map 2:v -c:v copy rear.mkv
```
The annotated stream includes bounding boxes for detected objects
(person, car, bus, truck, bicycle, and other classes)
and a live telemetry overlay (GPS, altitude, speed, heading).
> [!NOTE]
> Other ways to view the stream:
>
> - Leverage versatile streaming media players such as VLC Player to seamlessly handle, manage, and playback the incoming streams with ease and efficiency.
#### View in the ground-station dashboard (optional)
The `uav-ground-station` web dashboard shows every camera stream, the scene captions and the
CPU/GPU/NPU load of this host in one page. Run it on a **separate** machine from the UAV SDK
companion-compute board:
```bash
cd ../uav-ground-station
make up
```
Then open `http://:8080`, enter the **UAV host's IP** (`` above) and
press **Connect**. The camera cards appear once the cameras run (after the drone arms). See
the [UAV Ground Station README](https://github.com/open-edge-platform/federal-and-aerospace-ai-suite/blob/release-2026.3.0/uav-ground-station/README.md).
> [!IMPORTANT]
> If you do need to run the dashboard on the **same** host as
> `uav-mission-compute-sdk` (e.g., a single-machine test), it will conflict
> on port `8080`: the SDK's own REST API (`arm`/`takeoff`/`land`/`goto`)
> already binds `localhost:8080` on that host, which is also
> `uav-ground-station`'s default `VIZ_PORT`. Before `make up`, set a
> different port in `uav-ground-station/.env` (`make init` creates it first):
> `VIZ_PORT=8081`, then open `http://:8081` instead.
>
> The SDK's own `mediamtx` (the sim's camera-streaming container) also
> publishes WebRTC on `8889` + ICE on `8189` to the host, the same defaults
> `uav-ground-station`'s own `mediamtx` (`MEDIAMTX_WEBRTC_PORT` /
> `MEDIAMTX_WEBRTC_ICE_UDP_PORT` / `MEDIAMTX_WEBRTC_ICE_TCP_PORT`) uses —
> a single-machine test will fail to start with a `port is already
> allocated` error unless one side is moved; to solve this, you can add to
> `uav-ground-station/.env`: `MEDIAMTX_WEBRTC_PORT=8890` and
> `MEDIAMTX_WEBRTC_ICE_UDP_PORT=8190` / `MEDIAMTX_WEBRTC_ICE_TCP_PORT=8190`.
>
> `make up` shows video, captions, Flight Summary, and per-host metrics only.
> For the dashboard's **Search Videos** button and natural-language search
> over indexed footage, run `make up-vss` instead of `make up` (it brings up
> everything `make up` does, plus 8 extra VSS containers) — it needs its own
> required `.env` variables first (MinIO/Postgres credentials, an embedding
> model, and a detection model). See [VSS search in the UAV Ground Station Get Started guide](https://github.com/open-edge-platform/federal-and-aerospace-ai-suite/blob/release-2026.3.0/uav-ground-station/docs/user-guide/get-started.md#vss-search),
> and [tune the VSS search results](../how-to-customize/search-tuning/vss-search-tuning.md)
> once it is running.
**Stop an individual pipeline** (only needed if you started one manually via Option B in [Step 6](#6-start-inference-pipelines)):
```bash
curl -k -X DELETE https:///pipelines/${INSTANCE_ID}
```
### 8. Stop all services
Stop and remove the uav-vision-analytics stack (also removes named volumes):
```bash
make uavsdk-down
```
Then stop the SDK's core infrastructure:
```bash
cd ../uav-mission-compute-sdk
make down
```
---
## Pipelines
### UAV Mission Compute SDK Mode (`config-uavsdk.json`)
Nine pipelines with the same names as the simulator stack: `uav_object_detection_` (`cpu`, `gpu`, `npu`) and `uav_object_detection__` (`vlm` = `cpu` or `gpu`, with captions on MQTT `uav/caption/`). Their source is `rtsp_url`. `configs/cameras-uavsdk.yaml` picks them:
| Camera | Pipeline | Source (inside Docker) | Output RTSP (host) |
| --- | --- | --- | --- |
| `nadir_camera` | `uav_object_detection_cpu_gpu` | `rtsp://host.docker.internal:8554//nadir` | `rtsp://:8555/uav-nadir-camera` |
| `forward_camera` | `uav_object_detection_gpu_gpu` | `rtsp://host.docker.internal:8554//forward` | `rtsp://:8555/uav-forward-camera` |
| `rear_camera` | `uav_object_detection_npu_gpu` | `rtsp://host.docker.internal:8554//rear` | `rtsp://:8555/uav-rear-camera` |
> [!NOTE]
> `` is the `UAV_ID` environment variable (default: `uav-1`); set another value in `.env` if your SDK project uses a different vehicle ID. The cameras file uses the placeholder `{uav_id}`, so nothing else has to change.
All pipelines are `auto_start: false` — started by the manager (`make start-rtsp`) or through the REST API.
REST endpoint: `POST https:///pipelines/user_defined_pipelines/{name}` (proxied by nginx over HTTPS to `dlstreamer-pipeline-server:8081`)
---
## Telemetry Overlay Fields
Each output frame carries these overlaid fields in the upper-left corner:
| Field | Source MAVLink message | Description |
| --- | --- | --- |
| `Name` | — | Name passed as argument to the gvapython |
| `Frame` | — | Running frame counter |
| `ALT` | `GLOBAL_POSITION_INT.relative_alt` | Relative altitude (m) |
| `SPD` | `VFR_HUD.groundspeed` | Ground speed (m/s) |
| `HDG` | `GLOBAL_POSITION_INT.hdg` | Heading (degrees) |
| `LAT` | `GPS_RAW_INT.lat` | Latitude |
| `LON` | `GPS_RAW_INT.lon` | Longitude |
| `SATS` | `GPS_RAW_INT.satellites_visible` | GPS satellites visible |
---
## Port Reference
`nginx` is the only service that publishes ports to the host (bound to `HOST_IP`, not
`0.0.0.0`). `dlstreamer-pipeline-server` is reachable only on the internal `app_network`.
| Port | Protocol | Service | Published to host? | Mode |
| --- | --- | --- | --- | --- |
| `80` | HTTP | `nginx` → `301` redirect to HTTPS (no application traffic) | Yes | All modes |
| `443` | HTTPS | `nginx` → proxies to `dlstreamer-pipeline-server:8081` (REST API); self-signed cert, use `curl -k` | Yes | All modes |
| `8555` | RTSP | `nginx` → raw TCP passthrough to `dlstreamer-pipeline-server:8555` | Yes | All modes |
| `8081` | HTTP | DL Streamer REST API | No (internal only) | All modes |
---
## Documentation
| Document | Description |
| --- | --- |
| [index.md](../index.md) | Application overview and component block diagrams |
| [how-to-guides.md](../how-to-guides.md) | Operating the application: QGroundControl, ground station, deployment modes |
| [how-to-customize.md](../how-to-customize.md) | Changing cameras, models, pipelines, overlay and search tuning |
| [realsense-guide.md](../how-to-customize/camera-management/realsense-guide.md) | RealSense camera setup and pipelines |
| [benchmark.md](../benchmark.md) | Performance benchmarking guide |
| [makefile.md](../how-to-guides/makefile.md) | Makefile target reference |
| [troubleshooting.md](../how-to-guides/troubleshooting.md) | Known issues and resolutions |