Use MCP Agent Tools

The Web UI exposes an MCP server so an LLM-based agent can read what is on the signage display and drive it directly, alongside the existing camera-driven ad flow.

Endpoint

The MCP server runs inside the web-ui container on port 5100 (Streamable HTTP transport) and is reached through the nginx proxy at:

https://<HOST_IP>:5000/mcp

Port 5100 is not published directly on the host; all access goes through the TLS-terminating nginx proxy.

The nginx certificate is self-signed and regenerated on every nginx container restart (configs/nginx/nginx-cert-gen.sh). Its Subject Alternative Names cover localhost, 127.0.0.1, host.docker.internal, and HOST_IP (if set to a real IP/hostname in .env) — connect using one of those, or add another SAN entry for your case. An agent connecting from a container on the same host (e.g. via host.docker.internal) can trust this certificate automatically at its own startup using a trust-on-first-use fetch; see the qsr-agent project’s docs/architecture.md (“Remote MCP TLS trust”) for an example of that pattern.

Available Tools

Tool	Description
describe	Describes what the agent can read and do.
get_current_ad	Returns a short description of the advertisement currently shown.
get_catalog	Lists available products and their cross-sell promos.
select_dynamic_ad	Resolves a shopping context (weather, demand, age_mix, daypart) to a product and displays its ad.
trigger_ad	Directly displays a specific catalog item’s ad by name, bypassing context resolution. Like select_dynamic_ad, it generates a dynamic (AI-generated) ad when no predefined image exists for the item.
trigger_video_ad	Displays a looping video ad, live on screen for VIDEO_AD_DISPLAY_SECONDS (default 5s). Takes exactly one of item (a catalog product; uses its predefined video if provisioned, otherwise generates one with the catalog’s price/promo/slogan/frame overlays) or description (free text describing the video’s content directly, no overlays) — not both, since their overlays/content would be unrelated. MCP-triggered only — never produced by the camera-driven flow.
clear_ad	Clears any active agent-commanded override and returns to the camera-driven flow.

Video ad generation time: trigger_video_ad returns immediately and the video is generated in the background. On an Intel iGPU with the default f16 precision this takes about 25-30 s per video; the first request also loads the video model (about 20 s) unless AIG_PRELOAD_VIDEO_MODEL=true. Running in f32 takes about 3 minutes. See Change Inference Device. A new trigger_video_ad call may be rejected until the current video finishes.

Connect and Test

MCP Inspector

Requires Node.js (install via nvm if you don’t already have it).

NODE_TLS_REJECT_UNAUTHORIZED=0 npx @modelcontextprotocol/inspector

In the Inspector UI:

• Transport: Streamable HTTP

• URL: https://<HOST_IP>:5000/mcp

• Allow insecure/self-signed certificates (the nginx proxy uses a self-signed certificate)

Use “List Tools” to see the tools above, then call them directly to verify responses.

Python Client

pip install mcp==1.29.1 httpx==0.28.1

import asyncio
import ssl
import httpx
from mcp import ClientSession
from mcp.client.streamable_http import streamable_http_client


def get_ssl_context() -> ssl.SSLContext:
    ctx = ssl.create_default_context()
    ctx.check_hostname = False
    ctx.verify_mode = ssl.CERT_NONE
    return ctx


async def run_mcp_client(host_ip: str, port: int = 5000):
    uri = f"https://{host_ip}:{port}/mcp"
    ssl_context = get_ssl_context()

    print(f"Connecting to {uri}")

    # Build httpx client configured with our custom SSL context
    http_client = httpx.AsyncClient(
        verify=ssl_context,
        timeout=httpx.Timeout(30.0, read=300.0),
        follow_redirects=True,
    )

    try:
        async with http_client:
            async with streamable_http_client(uri, http_client=http_client) as (read_stream, write_stream, _):
                async with ClientSession(read_stream, write_stream) as session:
                    print("Initializing session...")
                    await session.initialize()

                    print("\nListing tools...")
                    tools_response = await session.list_tools()
                    print(tools_response)

                    print("\nCalling trigger_ad...")
                    result = await session.call_tool("trigger_ad", {"item": "banana"})
                    print("\nResult:")
                    print(result)

    except Exception as e:
        print(f"Error: {e}")


if __name__ == "__main__":
    asyncio.run(run_mcp_client("localhost"))

Notes

• If the mcp package fails to import or the server can’t bind its port, the MCP server logs an error and stays disabled; the rest of the application (Flask, MQTT, AIG) keeps running unaffected.

• select_dynamic_ad requires web-ui/context_rules.json to be configured; if it is missing or invalid, the tool is unavailable.