# MCP Server Architecture
Technical reference for how the Digital Signage MCP server is implemented, how a client
discovers and calls its tools, and how each tool relates to existing application code.
See [use-mcp-agent-tools.md](use-mcp-agent-tools.md) for end-user connection instructions.
## Overview
The MCP server ([web-ui/mcp_server.py](https://github.com/intel-retail/digital-signage/blob/release-2026.3.0/web-ui/mcp_server.py)) runs in-process
alongside the Flask app in the `web-ui` container, on port `5100`, using the `FastMCP`
Streamable HTTP transport. It is started as a daemon thread from
[web-ui/main.py](https://github.com/intel-retail/digital-signage/blob/release-2026.3.0/web-ui/main.py) after `initialize_app()` completes, so a
failure to import `mcp` or bind the port only disables the MCP thread — Flask, MQTT,
and AIG keep running.
Because it runs in the same process as `main.py`, tool implementations call shared
in-memory state directly (via `sys.modules['__main__']`) instead of making HTTP
requests to the REST API.
Externally, the server is reached through the nginx proxy at `/mcp`
(`configs/nginx/nginx.conf`), which forwards to `http://web-ui:5100` and terminates TLS.
## How a client discovers and calls tools
This follows the standard MCP protocol handshake — no custom discovery code:
1. **Registration**: each Python function decorated with `@mcp.tool()` in
`mcp_server.py` is auto-registered by the `FastMCP` SDK. The tool name is the
function name; the input schema is derived from the function's type-annotated
parameters; the description shown to clients is the function's docstring.
2. **`initialize`**: client opens a Streamable HTTP connection to `/mcp` and sends
`initialize`.
3. **`tools/list`**: client requests the tool catalog; the server returns all
registered tools with their descriptions and JSON-schema parameters.
4. **`tools/call`**: client invokes a tool by name with arguments; FastMCP dispatches
to the corresponding Python function and returns its string return value as the
result.
The `describe` tool exists as a convenience for agents that prefer reading a single
natural-language capability summary instead of parsing the schema.
## Tools added
| Tool | Signature | Purpose |
| --- | --- | --- |
| `describe()` | → str | Static capability summary for the agent |
| `get_current_ad()` | → str | What's currently on screen (camera / agent / generating) |
| `get_catalog()` | → str | Lists products and their cross-sell promos |
| `select_dynamic_ad()` | `(weather, demand, age_mix, daypart, display_seconds, benchmark)` → str | Resolves a shopping context to a product and displays its ad |
| `trigger_ad()` | `(item, display_seconds, promo_text, slogan)` → str | Directly displays a named catalog item's ad |
| `trigger_video_ad()` | `(item, description)` → str | Displays a looping video ad, live on screen for `VIDEO_AD_DISPLAY_SECONDS` (configured default 3s); exactly one of `item` (a catalog product; predefined video if provisioned, else AI-generated with catalog overlays) or `description` (free text describing the video directly) is required — not both, since their overlays/content would be unrelated; MCP-triggered only; generation runs in the background (about 25-30 s on GPU with the default `f16` precision, see [Change Inference Device](change-inference-device.md#video-precision-and-preloading)) |
| `clear_ad()` | → str | Clears the agent override, returns to camera-driven flow |
## New implementation vs. reuse of existing code
| Tool | Underlying code | New vs. reused |
| --- | --- | --- |
| `describe` | Inline string, no backing function | Fully new, MCP-only |
| `get_current_ad` | `get_active_ad_info()` | New function, but reads the existing `agent_override_*` / `last_selected_item` state — the same state `Ad_Generator.get_current_advertisement()` (used by the existing `/get_current_advertisement` REST route) already checks. Also reports the new `video_override_*` state (see `trigger_video_ad`) |
| `get_catalog` | `get_catalog_summary()` | New function built entirely on the existing `product_associations` dict, already populated from `ProductAssociations.csv` for the camera-driven flow |
| `trigger_ad` | `trigger_ad_core()` | New orchestration function; calls the pre-existing `resolve_product_label()` and the pre-existing core engine `Ad_Generator.generate_advertisement()` (same method the MQTT/camera pipeline uses). Introduces new state: `agent_override_ad/_until/_item/_generating`, `override_epoch` |
| `select_dynamic_ad` | `select_dynamic_ad_core()` + `resolve_context_to_product()` + `load_context_rules()` | New feature end-to-end (new `context_rules.json` config, new matching logic), but delegates final ad rendering to the same existing `generate_advertisement()` engine |
| `trigger_video_ad` | `trigger_video_ad_core()` + `Ad_Generator.generate_video_ad()` | Fully new, MCP-only video path with its own isolated state (`video_override_media/_until/_item/_generating`, `_video_generation_lock`). Predefined videos are looked up directly by product name (new `product_video_paths` dict from a new `pre_defined_ad_video` CSV column, mimetype detected from file extension) with no ChromaDB/AIG involvement. When no predefined video exists it calls a new AIG endpoint, `/aig/mvid/` (`Text2VideoPipeline`/LTX-Video model, separate from the existing image model), which generates genuinely temporally-related frames in a single call and returns an already-encoded looping animated WEBP (`image/webp`). AIG-side: new `videoinf.py` resource, new `AigServerMetadata` video-model config/singleton, and the promo/frame/logo overlay logic was extracted into a shared `apply_ad_overlays()` helper reused by both the image and video endpoints |
| `clear_ad` | `clear_agent_override()` | Fully new; manipulates the new override state fields |
**Notes:**
- Despite docstrings stating some core functions are "shared by the REST route and the
MCP tool," there is currently no REST route calling `trigger_ad_core` or
`select_dynamic_ad_core` — only `/`, `/portrait`, and `/get_current_advertisement`
exist as Flask routes today.
- The genuine reuse point across the camera-driven flow and the new MCP action tools
is `Ad_Generator.generate_advertisement()` — the image-generation engine that calls
the AIG server's `/ase/predef/query/ad` and `/aig/minf/` endpoints is unchanged in
interface and shared by both paths.
- On the AIG server side, the same change hardened (but did not add new endpoints to)
the existing `/aig/minf/` inference API: a `model_lock` was added for thread-safety
and the Flask app was switched to `threaded=True`, since MCP tools can now trigger
concurrent generation requests that didn't previously occur.
- `trigger_video_ad` reads its own `AIG_VIDEO_INFERENCE_DEVICE` env var (web-ui) to
pick the `device` sent to `/aig/mvid/`, kept deliberately separate from the image
path's `AIG_INFERENCE_DEVICE`. It must match the AIG server's `AIG_VIDEO_MODEL_DEVICE`
(same `.env` source variable) or the server can't reuse its preloaded video
pipeline and rebuilds it from disk on every call.
## Error reporting
The action tools (`select_dynamic_ad`, `trigger_ad`, `trigger_video_ad`) raise `ValueError` when the
core function returns an `error`, so FastMCP reports the call as a tool error (`isError`). Clients such
as the QSR agent rely on this to avoid recording a refused action as executed.
## Events
[web-ui/events.py](https://github.com/intel-retail/digital-signage/blob/release-2026.3.0/web-ui/events.py) builds the
`product_detected` and `ad_displayed` events, logs each as a `SIGNAGE_EVENT {json}` line and, when
`QSR_EVENTS_URL` is set, delivers it from a background thread. `main.py` emits `product_detected` from the
MQTT detection path (with a per-label cooldown) and `ad_displayed` when an ad goes live, from both the camera
flow and the agent tools. An image ad is announced when a display client is first served it, so an image
hidden behind a video override, superseded, or cleared before it is served produces no event. Event fields and settings are described in
[use-mcp-agent-tools.md](use-mcp-agent-tools.md#events).
## Control flow — client connecting and calling a tool
The full interaction is shown in four steps; each diagram continues from the one before it.
### 1. Session setup and tool discovery
```mermaid
---
config: {"theme":"base","themeVariables":{"actorBkg":"#E3F2FD","actorBorder":"#1E88E5","actorTextColor":"#0D47A1","signalColor":"#37474F","signalTextColor":"#263238","activationBkgColor":"#E8F5E9","activationBorderColor":"#43A047"},"sequence":{"mirrorActors":false,"messageAlign":"center"}}
---
sequenceDiagram
autonumber
participant Client as MCP Client
(Inspector / LLM agent)
participant Nginx as nginx proxy
(TLS, :5000)
participant MCP as MCP server thread
(FastMCP, :5100)
Client->>+Nginx: HTTPS POST /mcp (initialize)
Nginx->>+MCP: proxy_pass http://web-ui:5100/mcp
MCP-->>-Client: initialize result (capabilities)
deactivate Nginx
Client->>+Nginx: tools/list
Nginx->>+MCP: forward
MCP-->>-Client: [describe, get_current_ad, get_catalog,
select_dynamic_ad, trigger_ad, trigger_video_ad, clear_ad] + schemas
deactivate Nginx
```
### 2. `tools/call "trigger_ad"` — returns immediately
```mermaid
---
config: {"theme":"base","themeVariables":{"actorBkg":"#E3F2FD","actorBorder":"#1E88E5","actorTextColor":"#0D47A1","signalColor":"#37474F","signalTextColor":"#263238","activationBkgColor":"#E8F5E9","activationBorderColor":"#43A047"},"sequence":{"mirrorActors":false,"messageAlign":"center"}}
---
sequenceDiagram
autonumber
participant Client as MCP Client
(Inspector / LLM agent)
participant Nginx as nginx proxy
(TLS, :5000)
participant MCP as MCP server thread
(FastMCP, :5100)
participant Main as main.py
(__main__ / Flask app)
Client->>+Nginx: tools/call "trigger_ad" {item, display_seconds}
Nginx->>+MCP: forward (proxy_buffering off, streamable-http)
MCP->>+Main: _app() -> sys.modules['__main__']
MCP->>Main: trigger_ad_core(item, display_seconds, ...)
Main->>Main: resolve_product_label(item)
Main->>Main: acquire _generation_lock (non-blocking)
Main-->>-MCP: {'status': 'generating', 'item': resolved, ...}
MCP-->>-Client: "Ad for '- ' is generating now..." (immediate)
deactivate Nginx
```
### 3. Background daemon thread (async, after the response is sent)
```mermaid
---
config: {"theme":"base","themeVariables":{"actorBkg":"#E3F2FD","actorBorder":"#1E88E5","actorTextColor":"#0D47A1","signalColor":"#37474F","signalTextColor":"#263238","activationBkgColor":"#E8F5E9","activationBorderColor":"#43A047"},"sequence":{"mirrorActors":false,"messageAlign":"center"}}
---
sequenceDiagram
autonumber
participant Main as main.py
(__main__ / Flask app)
participant AdGen as Ad_Generator
(generate_advertisement)
participant AIG as AIG server
(:5003)
activate Main
Main->>+AdGen: generate_advertisement(item, associations)
AdGen->>+AIG: POST /ase/predef/query/ad (check predefined)
AIG-->>-AdGen: predefined image (if found)
alt no predefined match
AdGen->>+AIG: POST /aig/minf/ (dynamic generation)
AIG-->>-AdGen: generated image bytes
end
AdGen->>-Main: last_generated_ad captured
Main->>Main: agent_override_ad = captured
agent_override_until = now + secs
Main->>Main: release _generation_lock
deactivate Main
```
### 4. `tools/call "get_current_ad"` — reads the override state
```mermaid
---
config: {"theme":"base","themeVariables":{"actorBkg":"#E3F2FD","actorBorder":"#1E88E5","actorTextColor":"#0D47A1","signalColor":"#37474F","signalTextColor":"#263238","activationBkgColor":"#E8F5E9","activationBorderColor":"#43A047"},"sequence":{"mirrorActors":false,"messageAlign":"center"}}
---
sequenceDiagram
autonumber
participant Client as MCP Client
(Inspector / LLM agent)
participant Nginx as nginx proxy
(TLS, :5000)
participant MCP as MCP server thread
(FastMCP, :5100)
participant Main as main.py
(__main__ / Flask app)
Client->>+Nginx: tools/call "get_current_ad"
Nginx->>+MCP: forward
MCP->>+Main: get_active_ad_info()
Main-->>-MCP: {'mode': 'agent', 'item': ..., 'seconds_remaining': ...}
MCP-->>-Client: "Currently showing an agent-commanded ad for '- '..."
deactivate Nginx
```
## Data flow — state and payloads across components
The data flow is shown in four views that share the same components: how requests reach the tools, which tools read and write the shared state, the image-ad generation path, and the video-ad path.
### Request path — client to MCP tools and REST routes
```mermaid
---
config: {"theme":"base","themeVariables":{"fontSize":"18px","primaryColor":"#FFFFFF","primaryBorderColor":"#607D8B","primaryTextColor":"#263238","lineColor":"#546E7A","clusterBkg":"#FAFAFA","clusterBorder":"#B0BEC5"},"flowchart":{"curve":"linear","nodeSpacing":30,"rankSpacing":60,"padding":16}}
---
flowchart TB
classDef client fill:#E3F2FD,stroke:#1E88E5,stroke-width:2px,color:#0D47A1
classDef proxy fill:#FFF3E0,stroke:#FB8C00,stroke-width:2px,color:#E65100
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef flask fill:#E0F7FA,stroke:#00ACC1,stroke-width:2px,color:#006064
subgraph Client["MCP Client"]
C1["LLM agent / Inspector"]
end
subgraph Proxy["nginx (TLS termination)"]
N1["/mcp location
proxy_pass web-ui:5100"]
end
subgraph WebUI["web-ui container"]
subgraph MCPThread["MCP server thread (:5100)"]
direction LR
T1["describe"]
T2["get_current_ad"]
T3["get_catalog"]
T4["select_dynamic_ad"]
T5["trigger_ad"]
T6["clear_ad"]
T7["trigger_video_ad"]
end
F1["Flask routes
/, /portrait,
/get_current_advertisement"]
end
C1 <==>|"HTTPS: initialize / tools-list / tools-call"| N1
N1 <==>|"HTTP proxy"| MCPThread
C1 -.->|"REST (unrelated path)"| F1
class C1 client
class N1 proxy
class T1,T2,T3,T4,T5,T6,T7 tool
class F1 flask
```
### Image ad state — which tools read and write the shared state
```mermaid
---
config: {"theme":"base","themeVariables":{"fontSize":"18px","primaryColor":"#FFFFFF","primaryBorderColor":"#607D8B","primaryTextColor":"#263238","lineColor":"#546E7A","clusterBkg":"#FAFAFA","clusterBorder":"#B0BEC5"},"flowchart":{"curve":"linear","nodeSpacing":24,"rankSpacing":140,"padding":16}}
---
flowchart LR
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef state fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px,color:#4A148C
classDef lock fill:#FCE4EC,stroke:#D81B60,stroke-width:2px,stroke-dasharray:5 3,color:#880E4F
classDef flask fill:#E0F7FA,stroke:#00ACC1,stroke-width:2px,color:#006064
subgraph WebUI["web-ui container"]
direction LR
subgraph MCPThread["MCP server thread (:5100)"]
direction TB
T2["get_current_ad"]
T3["get_catalog"]
T4["select_dynamic_ad"]
T5["trigger_ad"]
T6["clear_ad"]
end
F1["Flask routes
/, /portrait,
/get_current_advertisement"]
MQ["MQTT subscriber
(camera detections)"]
subgraph SharedState["Shared in-memory state (Ad_Generator instance)"]
direction TB
S1[("product_associations
from ProductAssociations.csv")]
S2[("context_rules
from context_rules.json")]
S3[("agent_override_ad /
_until / _item / _generating
override_epoch")]
S4[("last_generated_ad
last_selected_item")]
S5[("_generation_lock")]
end
end
T2 --> S3
T2 --> S4
T3 --> S1
T4 --> S2
T4 --> S1
T4 -->|"acquire/release"| S5
T5 --> S1
T5 -->|"acquire/release"| S5
T6 --> S3
MQ -->|"camera-detected item"| S1
F1 -->|"reads"| S3
F1 -->|"reads"| S4
class T2,T3,T4,T5,T6 tool
class S1,S2,S3,S4 state
class S5 lock
class F1,MQ flask
```
### Image ad generation — from the tools through AIG into the override state
```mermaid
---
config: {"theme":"base","themeVariables":{"fontSize":"18px","primaryColor":"#FFFFFF","primaryBorderColor":"#607D8B","primaryTextColor":"#263238","lineColor":"#546E7A","clusterBkg":"#FAFAFA","clusterBorder":"#B0BEC5"},"flowchart":{"curve":"linear","nodeSpacing":30,"rankSpacing":60,"padding":16}}
---
flowchart TB
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef state fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px,color:#4A148C
classDef flask fill:#E0F7FA,stroke:#00ACC1,stroke-width:2px,color:#006064
classDef aig fill:#FFFDE7,stroke:#F9A825,stroke-width:2px,color:#F57F17
subgraph Callers["web-ui container — callers of generate_advertisement"]
direction LR
T4["select_dynamic_ad"]
T5["trigger_ad"]
MQ["MQTT subscriber
(camera detections)"]
end
subgraph AIGServer["aig-server container (:5003)"]
direction LR
A1["/ase/predef/query/ad
(predefined lookup)"]
A2["/aig/minf/
(dynamic image generation, model_lock)"]
end
subgraph SharedState["Shared in-memory state (Ad_Generator instance)"]
direction LR
S3[("agent_override_ad /
_until / _item / _generating
override_epoch")]
S4[("last_generated_ad
last_selected_item")]
end
T4 & T5 -->|"generate_advertisement"| A1
MQ -->|"generate_advertisement when no override"| A1
A1 -->|"miss"| A2
A1 & A2 -->|"image bytes"| S3
S3 -->|"captured ad, expiry timestamp"| S4
class T4,T5 tool
class S3,S4 state
class MQ flask
class A1,A2 aig
```
### Video ad path — `trigger_video_ad`, isolated video state and the AIG video endpoint
```mermaid
---
config: {"theme":"base","themeVariables":{"fontSize":"18px","primaryColor":"#FFFFFF","primaryBorderColor":"#607D8B","primaryTextColor":"#263238","lineColor":"#546E7A","clusterBkg":"#FAFAFA","clusterBorder":"#B0BEC5"},"flowchart":{"curve":"linear","nodeSpacing":30,"rankSpacing":60,"padding":16}}
---
flowchart TB
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef state fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px,color:#4A148C
classDef lock fill:#FCE4EC,stroke:#D81B60,stroke-width:2px,stroke-dasharray:5 3,color:#880E4F
classDef aig fill:#FFFDE7,stroke:#F9A825,stroke-width:2px,color:#F57F17
subgraph WebUI["web-ui container"]
subgraph MCPThread["MCP server thread (:5100)"]
direction LR
T2["get_current_ad"]
T6["clear_ad"]
T7["trigger_video_ad"]
end
subgraph SharedState["Shared in-memory state (Ad_Generator instance)"]
S1[("product_associations
from ProductAssociations.csv")]
S6[("video_override_media /
_until / _item / _generating
product_video_paths")]
S7[("_video_generation_lock")]
end
end
subgraph AIGServer["aig-server container (:5003)"]
A3["/aig/mvid/
(video generation, LTX-Video)"]
end
T2 --> S6
T6 --> S6
T7 --> S1
T7 -->|"acquire/release"| S7
T7 -->|"predefined video miss"| A3
A3 -->|"animated WEBP"| S6
class T2,T6,T7 tool
class S1,S6 state
class S7 lock
class A3 aig
```
**Key control-flow points:**
- All 7 tools are dispatched through the single FastMCP thread; each calls back into
`main.py`'s module-level functions via `_app()` (same process, no HTTP hop internally).
- `trigger_ad` / `select_dynamic_ad` return immediately (status `generating`) while the
actual image generation runs in a separate daemon thread — the MCP call never blocks
on the AIG round-trip (except when `benchmark=True`).
- `_generation_lock` + `override_epoch` prevent two concurrent generations (agent vs.
camera-driven, or agent vs. a `clear_ad` mid-flight) from corrupting shared state.
**Key data-flow points:**
- `agent_override_*` fields are the single hand-off point between MCP tools and the
display: both the REST `/get_current_advertisement` route and MCP's `get_current_ad`
read the same override state.
- `product_associations` (CSV) and `context_rules` (JSON) are load-once, read-many
configuration feeding both the camera-driven MQTT path and the new MCP tools.