Use the multimodal MCP server#
The optional Model Context Protocol (MCP) server exposes the weld defect detection sample app to MCP clients.
It uses Python/FastMCP and Streamable HTTP through the existing HTTPS gateway at /mcp.
Grafana, the data ingestion paths, and the agentic UI remain unchanged.
Prerequisites#
Complete the Get Started prerequisites.
Meet the requirements of the agentic workflow, including the VLM model download. The MCP server is deployed only with the agentic stack.
Use Python 3.12 or later for the example client.
Set
HOST_IPin.envto the gateway’s IPv4 address.
Deploy#
Caution
make up_agentic stops the existing stack and removes its volumes.
Use it only on a deployment that you intend to replace.
Set MCP credentials in
.env:MCP_USERNAME=mcpuser MCP_PASSWORD=<your-password>
The username must contain only alphabets and be at least 5 characters long. The password must be at least 10 alphanumeric characters with at least one letter and one digit.
make check_env_variables MCP=truevalidates both values. Never commit credentials.Build the images:
make buildmake buildalso builds the MCP server image; a published MCP image is not required.DOCKER_REGISTRYandMULTIMODAL_MCP_SERVER_IMAGEcontrol its registry and repository. Runtime package versions are pinned inmcp-server/requirements.txt.Deploy the agentic stack with the MCP server:
make up_agentic MCP=true
Pipelines do not start automatically in this mode. The time-series UDF package is not uploaded during deployment either. Both are done by the
start_pipelinetool. WithoutMCP=true,make up_agenticretains the automatic startup and does not deploy the MCP server.
Trust the gateway certificate#
All MCP requests require Basic authentication over verified HTTPS, including initialization and discovery. There is no insecure client option.
The local gateway creates a self-signed certificate with localhost and HOST_IP subject alternative names.
Obtain this certificate through a trusted local connection:
docker exec nginx_proxy cat /opt/nginx/certs/cert.pem > ./mcp-gateway-ca.pem
export MCP_CA_FILE="$PWD/mcp-gateway-ca.pem"
Store the certificate outside version control. The gateway regenerates its certificate on restart; obtain the new certificate after a restart. For production, provision an organization-trusted certificate and key instead of the local certificate-generation step.
Use the example client#
The client reads MCP_USERNAME, MCP_PASSWORD, HOST_IP, and GRAFANA_PORT from the component .env
and connects to https://HOST_IP:GRAFANA_PORT/mcp by default.
Environment variables, including MCP_URL and MCP_CA_FILE, override values from .env.
Run the client with --help to see all options; --url overrides MCP_URL and --ca-file overrides MCP_CA_FILE.
From the component root:
python3 -m venv mcp-server/example/.venv
mcp-server/example/.venv/bin/pip install -r mcp-server/example/requirements.txt
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py describe
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py get_pipeline_status
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py start_pipeline \
--arguments '{"vision_inf_device":"CPU","time_series_inf_device":"CPU"}'
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py list_insights_data \
--arguments '{"page":1,"page_size":10}'
# Select one of the timestamps from the above results as the value in the place of <timestamp> in the command below
# Please note the vllm-server container would take around 10 mins to initialize fully, try after this time to run the command below
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py explain \
--arguments '{"selected_time":"<timestamp>"}'
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py run_agent \
--arguments '{"time_range":"30s"}'
# Select the run id from the previous result and use it as the value in the place of <run-id>
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py get_run_results \
--arguments '{"run_id":"<run-id>"}'
mcp-server/example/.venv/bin/python mcp-server/example/example_client.py stop_pipeline
Use a timestamp from the list_insights_data results for selected_time, and the run ID returned by run_agent for run_id.
start_pipeline selects the vision device (CPU, GPU, or NPU) and the time-series device (CPU or GPU) independently.
The MCP container itself does not require a GPU or NPU.
Caution
start_pipeline, stop_pipeline, and run_agent affect live processing.
Use them only on a deployment that you intend to control.
Tools#
Tool |
Result |
|---|---|
|
Registered tools, input schemas, and operational guidance |
|
Uploads the time-series UDF package, applies its config, waits until the task is enabled, then starts the weld classification pipeline |
|
Stops all queued or running DL Streamer pipelines and disables the weld time-series task |
|
Time-series task status and DL Streamer pipeline statuses |
|
Paginated fusion results ( |
|
Explanation of one fused detection, selected by its ISO-8601 timestamp |
|
Starts an agentic analysis over |
|
Status of an agent run ( |
All MCP requests require the Basic-auth credentials from .env, including initialization and discovery.
Requests without valid credentials receive HTTP 401.
Partial failures in start_pipeline and stop_pipeline list the stages that completed and the ones that failed.
explain and run_agent depend on the vLLM and agentic services.
Agent and model-generated analysis is advisory; review results before making operational or safety decisions.
The server does not expose arbitrary commands, queries, or uploads from clients. Upstream errors are returned as sanitized tool failures.
Validate and stop#
make status
make down MCP=true
Use MCP=true for down so the MCP server is removed with the rest of the stack.
The server port is internal; only nginx exposes /mcp.