MCP Server Architecture#
Technical reference for how the Digital Signage MCP server is implemented, how a client discovers and calls its tools, and how each tool relates to existing application code. See use-mcp-agent-tools.md for end-user connection instructions.
Overview#
The MCP server (web-ui/mcp_server.py) runs in-process
alongside the Flask app in the web-ui container, on port 5100, using the FastMCP
Streamable HTTP transport. It is started as a daemon thread from
web-ui/main.py after initialize_app() completes, so a
failure to import mcp or bind the port only disables the MCP thread — Flask, MQTT,
and AIG keep running.
Because it runs in the same process as main.py, tool implementations call shared
in-memory state directly (via sys.modules['__main__']) instead of making HTTP
requests to the REST API.
Externally, the server is reached through the nginx proxy at /mcp
(configs/nginx/nginx.conf), which forwards to http://web-ui:5100 and terminates TLS.
How a client discovers and calls tools#
This follows the standard MCP protocol handshake — no custom discovery code:
Registration: each Python function decorated with
@mcp.tool()inmcp_server.pyis auto-registered by theFastMCPSDK. The tool name is the function name; the input schema is derived from the function’s type-annotated parameters; the description shown to clients is the function’s docstring.initialize: client opens a Streamable HTTP connection to/mcpand sendsinitialize.tools/list: client requests the tool catalog; the server returns all registered tools with their descriptions and JSON-schema parameters.tools/call: client invokes a tool by name with arguments; FastMCP dispatches to the corresponding Python function and returns its string return value as the result.
The describe tool exists as a convenience for agents that prefer reading a single
natural-language capability summary instead of parsing the schema.
Tools added#
Tool |
Signature |
Purpose |
|---|---|---|
|
→ str |
Static capability summary for the agent |
|
→ str |
What’s currently on screen (camera / agent / generating) |
|
→ str |
Lists products and their cross-sell promos |
|
|
Resolves a shopping context to a product and displays its ad |
|
|
Directly displays a named catalog item’s ad |
|
|
Displays a looping video ad, live on screen for |
|
→ str |
Clears the agent override, returns to camera-driven flow |
New implementation vs. reuse of existing code#
Tool |
Underlying code |
New vs. reused |
|---|---|---|
|
Inline string, no backing function |
Fully new, MCP-only |
|
|
New function, but reads the existing |
|
|
New function built entirely on the existing |
|
|
New orchestration function; calls the pre-existing |
|
|
New feature end-to-end (new |
|
|
Fully new, MCP-only video path with its own isolated state ( |
|
|
Fully new; manipulates the new override state fields |
Notes:
Despite docstrings stating some core functions are “shared by the REST route and the MCP tool,” there is currently no REST route calling
trigger_ad_coreorselect_dynamic_ad_core— only/,/portrait, and/get_current_advertisementexist as Flask routes today.The genuine reuse point across the camera-driven flow and the new MCP action tools is
Ad_Generator.generate_advertisement()— the image-generation engine that calls the AIG server’s/ase/predef/query/adand/aig/minf/endpoints is unchanged in interface and shared by both paths.On the AIG server side, the same change hardened (but did not add new endpoints to) the existing
/aig/minf/inference API: amodel_lockwas added for thread-safety and the Flask app was switched tothreaded=True, since MCP tools can now trigger concurrent generation requests that didn’t previously occur.trigger_video_adreads its ownAIG_VIDEO_INFERENCE_DEVICEenv var (web-ui) to pick thedevicesent to/aig/mvid/, kept deliberately separate from the image path’sAIG_INFERENCE_DEVICE. It must match the AIG server’sAIG_VIDEO_MODEL_DEVICE(same.envsource variable) or the server can’t reuse its preloaded video pipeline and rebuilds it from disk on every call.
Error reporting#
The action tools (select_dynamic_ad, trigger_ad, trigger_video_ad) raise ValueError when the
core function returns an error, so FastMCP reports the call as a tool error (isError). Clients such
as the QSR agent rely on this to avoid recording a refused action as executed.
Events#
web-ui/events.py builds the
product_detected and ad_displayed events, logs each as a SIGNAGE_EVENT {json} line and, when
QSR_EVENTS_URL is set, delivers it from a background thread. main.py emits product_detected from the
MQTT detection path (with a per-label cooldown) and ad_displayed when an ad goes live, from both the camera
flow and the agent tools. An image ad is announced when a display client is first served it, so an image
hidden behind a video override, superseded, or cleared before it is served produces no event. Event fields and settings are described in
use-mcp-agent-tools.md.
Control flow — client connecting and calling a tool#
The full interaction is shown in four steps; each diagram continues from the one before it.
1. Session setup and tool discovery#
---
config:
sequence:
messageAlign: center
mirrorActors: false
theme: base
themeVariables:
activationBkgColor: '#E8F5E9'
activationBorderColor: '#43A047'
actorBkg: '#E3F2FD'
actorBorder: '#1E88E5'
actorTextColor: '#0D47A1'
signalColor: '#37474F'
signalTextColor: '#263238'
---
sequenceDiagram
autonumber
participant Client as MCP Client<br/>(Inspector / LLM agent)
participant Nginx as nginx proxy<br/>(TLS, :5000)
participant MCP as MCP server thread<br/>(FastMCP, :5100)
Client->>+Nginx: HTTPS POST /mcp (initialize)
Nginx->>+MCP: proxy_pass http://web-ui:5100/mcp
MCP-->>-Client: initialize result (capabilities)
deactivate Nginx
Client->>+Nginx: tools/list
Nginx->>+MCP: forward
MCP-->>-Client: [describe, get_current_ad, get_catalog,<br/>select_dynamic_ad, trigger_ad, trigger_video_ad, clear_ad] + schemas
deactivate Nginx
2. tools/call "trigger_ad" — returns immediately#
---
config:
sequence:
messageAlign: center
mirrorActors: false
theme: base
themeVariables:
activationBkgColor: '#E8F5E9'
activationBorderColor: '#43A047'
actorBkg: '#E3F2FD'
actorBorder: '#1E88E5'
actorTextColor: '#0D47A1'
signalColor: '#37474F'
signalTextColor: '#263238'
---
sequenceDiagram
autonumber
participant Client as MCP Client<br/>(Inspector / LLM agent)
participant Nginx as nginx proxy<br/>(TLS, :5000)
participant MCP as MCP server thread<br/>(FastMCP, :5100)
participant Main as main.py<br/>(__main__ / Flask app)
Client->>+Nginx: tools/call "trigger_ad" {item, display_seconds}
Nginx->>+MCP: forward (proxy_buffering off, streamable-http)
MCP->>+Main: _app() -> sys.modules['__main__']
MCP->>Main: trigger_ad_core(item, display_seconds, ...)
Main->>Main: resolve_product_label(item)
Main->>Main: acquire _generation_lock (non-blocking)
Main-->>-MCP: {'status': 'generating', 'item': resolved, ...}
MCP-->>-Client: "Ad for '<item>' is generating now..." (immediate)
deactivate Nginx
3. Background daemon thread (async, after the response is sent)#
---
config:
sequence:
messageAlign: center
mirrorActors: false
theme: base
themeVariables:
activationBkgColor: '#E8F5E9'
activationBorderColor: '#43A047'
actorBkg: '#E3F2FD'
actorBorder: '#1E88E5'
actorTextColor: '#0D47A1'
signalColor: '#37474F'
signalTextColor: '#263238'
---
sequenceDiagram
autonumber
participant Main as main.py<br/>(__main__ / Flask app)
participant AdGen as Ad_Generator<br/>(generate_advertisement)
participant AIG as AIG server<br/>(:5003)
activate Main
Main->>+AdGen: generate_advertisement(item, associations)
AdGen->>+AIG: POST /ase/predef/query/ad (check predefined)
AIG-->>-AdGen: predefined image (if found)
alt no predefined match
AdGen->>+AIG: POST /aig/minf/ (dynamic generation)
AIG-->>-AdGen: generated image bytes
end
AdGen->>-Main: last_generated_ad captured
Main->>Main: agent_override_ad = captured<br/>agent_override_until = now + secs
Main->>Main: release _generation_lock
deactivate Main
4. tools/call "get_current_ad" — reads the override state#
---
config:
sequence:
messageAlign: center
mirrorActors: false
theme: base
themeVariables:
activationBkgColor: '#E8F5E9'
activationBorderColor: '#43A047'
actorBkg: '#E3F2FD'
actorBorder: '#1E88E5'
actorTextColor: '#0D47A1'
signalColor: '#37474F'
signalTextColor: '#263238'
---
sequenceDiagram
autonumber
participant Client as MCP Client<br/>(Inspector / LLM agent)
participant Nginx as nginx proxy<br/>(TLS, :5000)
participant MCP as MCP server thread<br/>(FastMCP, :5100)
participant Main as main.py<br/>(__main__ / Flask app)
Client->>+Nginx: tools/call "get_current_ad"
Nginx->>+MCP: forward
MCP->>+Main: get_active_ad_info()
Main-->>-MCP: {'mode': 'agent', 'item': ..., 'seconds_remaining': ...}
MCP-->>-Client: "Currently showing an agent-commanded ad for '<item>'..."
deactivate Nginx
Data flow — state and payloads across components#
The data flow is shown in four views that share the same components: how requests reach the tools, which tools read and write the shared state, the image-ad generation path, and the video-ad path.
Request path — client to MCP tools and REST routes#
---
config:
flowchart:
curve: linear
nodeSpacing: 30
padding: 16
rankSpacing: 60
theme: base
themeVariables:
clusterBkg: '#FAFAFA'
clusterBorder: '#B0BEC5'
fontSize: 18px
lineColor: '#546E7A'
primaryBorderColor: '#607D8B'
primaryColor: '#FFFFFF'
primaryTextColor: '#263238'
---
flowchart TB
classDef client fill:#E3F2FD,stroke:#1E88E5,stroke-width:2px,color:#0D47A1
classDef proxy fill:#FFF3E0,stroke:#FB8C00,stroke-width:2px,color:#E65100
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef flask fill:#E0F7FA,stroke:#00ACC1,stroke-width:2px,color:#006064
subgraph Client["MCP Client"]
C1["LLM agent / Inspector"]
end
subgraph Proxy["nginx (TLS termination)"]
N1["/mcp location<br/>proxy_pass web-ui:5100"]
end
subgraph WebUI["web-ui container"]
subgraph MCPThread["MCP server thread (:5100)"]
direction LR
T1["describe"]
T2["get_current_ad"]
T3["get_catalog"]
T4["select_dynamic_ad"]
T5["trigger_ad"]
T6["clear_ad"]
T7["trigger_video_ad"]
end
F1["Flask routes<br/>/, /portrait,<br/>/get_current_advertisement"]
end
C1 <==>|"HTTPS: initialize / tools-list / tools-call"| N1
N1 <==>|"HTTP proxy"| MCPThread
C1 -.->|"REST (unrelated path)"| F1
class C1 client
class N1 proxy
class T1,T2,T3,T4,T5,T6,T7 tool
class F1 flask
Image ad generation — from the tools through AIG into the override state#
---
config:
flowchart:
curve: linear
nodeSpacing: 30
padding: 16
rankSpacing: 60
theme: base
themeVariables:
clusterBkg: '#FAFAFA'
clusterBorder: '#B0BEC5'
fontSize: 18px
lineColor: '#546E7A'
primaryBorderColor: '#607D8B'
primaryColor: '#FFFFFF'
primaryTextColor: '#263238'
---
flowchart TB
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef state fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px,color:#4A148C
classDef flask fill:#E0F7FA,stroke:#00ACC1,stroke-width:2px,color:#006064
classDef aig fill:#FFFDE7,stroke:#F9A825,stroke-width:2px,color:#F57F17
subgraph Callers["web-ui container — callers of generate_advertisement"]
direction LR
T4["select_dynamic_ad"]
T5["trigger_ad"]
MQ["MQTT subscriber<br/>(camera detections)"]
end
subgraph AIGServer["aig-server container (:5003)"]
direction LR
A1["/ase/predef/query/ad<br/>(predefined lookup)"]
A2["/aig/minf/<br/>(dynamic image generation, model_lock)"]
end
subgraph SharedState["Shared in-memory state (Ad_Generator instance)"]
direction LR
S3[("agent_override_ad /<br/>_until / _item / _generating<br/>override_epoch")]
S4[("last_generated_ad<br/>last_selected_item")]
end
T4 & T5 -->|"generate_advertisement"| A1
MQ -->|"generate_advertisement when no override"| A1
A1 -->|"miss"| A2
A1 & A2 -->|"image bytes"| S3
S3 -->|"captured ad, expiry timestamp"| S4
class T4,T5 tool
class S3,S4 state
class MQ flask
class A1,A2 aig
Video ad path — trigger_video_ad, isolated video state and the AIG video endpoint#
---
config:
flowchart:
curve: linear
nodeSpacing: 30
padding: 16
rankSpacing: 60
theme: base
themeVariables:
clusterBkg: '#FAFAFA'
clusterBorder: '#B0BEC5'
fontSize: 18px
lineColor: '#546E7A'
primaryBorderColor: '#607D8B'
primaryColor: '#FFFFFF'
primaryTextColor: '#263238'
---
flowchart TB
classDef tool fill:#E8F5E9,stroke:#43A047,stroke-width:2px,color:#1B5E20
classDef state fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px,color:#4A148C
classDef lock fill:#FCE4EC,stroke:#D81B60,stroke-width:2px,stroke-dasharray:5 3,color:#880E4F
classDef aig fill:#FFFDE7,stroke:#F9A825,stroke-width:2px,color:#F57F17
subgraph WebUI["web-ui container"]
subgraph MCPThread["MCP server thread (:5100)"]
direction LR
T2["get_current_ad"]
T6["clear_ad"]
T7["trigger_video_ad"]
end
subgraph SharedState["Shared in-memory state (Ad_Generator instance)"]
S1[("product_associations<br/>from ProductAssociations.csv")]
S6[("video_override_media /<br/>_until / _item / _generating<br/>product_video_paths")]
S7[("_video_generation_lock")]
end
end
subgraph AIGServer["aig-server container (:5003)"]
A3["/aig/mvid/<br/>(video generation, LTX-Video)"]
end
T2 --> S6
T6 --> S6
T7 --> S1
T7 -->|"acquire/release"| S7
T7 -->|"predefined video miss"| A3
A3 -->|"animated WEBP"| S6
class T2,T6,T7 tool
class S1,S6 state
class S7 lock
class A3 aig
Key control-flow points:
All 7 tools are dispatched through the single FastMCP thread; each calls back into
main.py’s module-level functions via_app()(same process, no HTTP hop internally).trigger_ad/select_dynamic_adreturn immediately (statusgenerating) while the actual image generation runs in a separate daemon thread — the MCP call never blocks on the AIG round-trip (except whenbenchmark=True)._generation_lock+override_epochprevent two concurrent generations (agent vs. camera-driven, or agent vs. aclear_admid-flight) from corrupting shared state.
Key data-flow points:
agent_override_*fields are the single hand-off point between MCP tools and the display: both the REST/get_current_advertisementroute and MCP’sget_current_adread the same override state.product_associations(CSV) andcontext_rules(JSON) are load-once, read-many configuration feeding both the camera-driven MQTT path and the new MCP tools.