# Use the multimodal MCP server The optional Model Context Protocol (MCP) server exposes the weld defect detection sample app to MCP clients. It uses Python/FastMCP and Streamable HTTP through the existing HTTPS gateway at `/mcp`. Grafana, the data ingestion paths, and the agentic UI remain unchanged. ## Prerequisites - Complete the [Get Started prerequisites](../get-started.md). - Meet the requirements of the [agentic workflow](./how-to-deploy-agent-workflow.md), including the VLM model download. The MCP server is deployed only with the agentic stack. - Use Python 3.12 or later for the example client. - Set `HOST_IP` in `.env` to the gateway's IPv4 address. ## Deploy > [!CAUTION] > `make up_agentic` stops the existing stack and removes its volumes. > Use it only on a deployment that you intend to replace. 1. Set MCP credentials in `.env`: ```bash MCP_USERNAME=mcpuser MCP_PASSWORD= ``` The username must contain only alphabets and be at least 5 characters long. The password must be at least 10 alphanumeric characters with at least one letter and one digit. `make check_env_variables MCP=true` validates both values. Never commit credentials. 2. Build the images: ```bash make build ``` `make build` also builds the MCP server image; a published MCP image is not required. `DOCKER_REGISTRY` and `MULTIMODAL_MCP_SERVER_IMAGE` control its registry and repository. Runtime package versions are pinned in `mcp-server/requirements.txt`. 3. Deploy the agentic stack with the MCP server: ```bash make up_agentic MCP=true ``` Pipelines do not start automatically in this mode. The time-series UDF package is not uploaded during deployment either. Both are done by the `start_pipeline` tool. Without `MCP=true`, `make up_agentic` retains the automatic startup and does not deploy the MCP server. ## Trust the gateway certificate All MCP requests require Basic authentication over verified HTTPS, including initialization and discovery. There is no insecure client option. The local gateway creates a self-signed certificate with `localhost` and `HOST_IP` subject alternative names. Obtain this certificate through a trusted local connection: ```bash docker exec nginx_proxy cat /opt/nginx/certs/cert.pem > ./mcp-gateway-ca.pem export MCP_CA_FILE="$PWD/mcp-gateway-ca.pem" ``` Store the certificate outside version control. The gateway regenerates its certificate on restart; obtain the new certificate after a restart. For production, provision an organization-trusted certificate and key instead of the local certificate-generation step. ## Use the example client The client reads `MCP_USERNAME`, `MCP_PASSWORD`, `HOST_IP`, and `GRAFANA_PORT` from the component `.env` and connects to `https://HOST_IP:GRAFANA_PORT/mcp` by default. Environment variables, including `MCP_URL` and `MCP_CA_FILE`, override values from `.env`. Run the client with `--help` to see all options; `--url` overrides `MCP_URL` and `--ca-file` overrides `MCP_CA_FILE`. From the component root: ```bash python3 -m venv mcp-server/example/.venv mcp-server/example/.venv/bin/pip install -r mcp-server/example/requirements.txt mcp-server/example/.venv/bin/python mcp-server/example/example_client.py describe mcp-server/example/.venv/bin/python mcp-server/example/example_client.py get_pipeline_status mcp-server/example/.venv/bin/python mcp-server/example/example_client.py start_pipeline \ --arguments '{"vision_inf_device":"CPU","time_series_inf_device":"CPU"}' mcp-server/example/.venv/bin/python mcp-server/example/example_client.py list_insights_data \ --arguments '{"page":1,"page_size":10}' # Select one of the timestamps from the above results as the value in the place of in the command below # Please note the vllm-server container would take around 10 mins to initialize fully, try after this time to run the command below mcp-server/example/.venv/bin/python mcp-server/example/example_client.py explain \ --arguments '{"selected_time":""}' mcp-server/example/.venv/bin/python mcp-server/example/example_client.py run_agent \ --arguments '{"time_range":"30s"}' # Select the run id from the previous result and use it as the value in the place of mcp-server/example/.venv/bin/python mcp-server/example/example_client.py get_run_results \ --arguments '{"run_id":""}' mcp-server/example/.venv/bin/python mcp-server/example/example_client.py stop_pipeline ``` Use a timestamp from the `list_insights_data` results for `selected_time`, and the run ID returned by `run_agent` for `run_id`. `start_pipeline` selects the vision device (`CPU`, `GPU`, or `NPU`) and the time-series device (`CPU` or `GPU`) independently. The MCP container itself does not require a GPU or NPU. > [!CAUTION] > `start_pipeline`, `stop_pipeline`, and `run_agent` affect live processing. > Use them only on a deployment that you intend to control. ## Tools | Tool | Result | | --- | --- | | `describe` | Registered tools, input schemas, and operational guidance | | `start_pipeline` | Uploads the time-series UDF package, applies its config, waits until the task is enabled, then starts the weld classification pipeline | | `stop_pipeline` | Stops all queued or running DL Streamer pipelines and disables the weld time-series task | | `get_pipeline_status` | Time-series task status and DL Streamer pipeline statuses | | `list_insights_data` | Paginated fusion results (`page`, `page_size`) | | `explain` | Explanation of one fused detection, selected by its ISO-8601 timestamp | | `run_agent` | Starts an agentic analysis over `30s`, `1m`, `5m`, `10m`, or `30m` and returns the run ID | | `get_run_results` | Status of an agent run (`reasoning`, `completed`, or `error`) and its result once completed | All MCP requests require the Basic-auth credentials from `.env`, including initialization and discovery. Requests without valid credentials receive HTTP 401. Partial failures in `start_pipeline` and `stop_pipeline` list the stages that completed and the ones that failed. `explain` and `run_agent` depend on the vLLM and agentic services. Agent and model-generated analysis is advisory; review results before making operational or safety decisions. The server does not expose arbitrary commands, queries, or uploads from clients. Upstream errors are returned as sanitized tool failures. ## Validate and stop ```bash make status make down MCP=true ``` Use `MCP=true` for `down` so the MCP server is removed with the rest of the stack. The server port is internal; only nginx exposes `/mcp`.