# How It Works This document provides a comprehensive technical overview of the system architecture, component interactions, data flows, and design decisions. ## System Architecture ### High-Level Architecture ![High-Level Architecture](./_assets/OA-DI-architecture.svg "dine-in high-level architecture") ### Request Flow ```mermaid sequenceDiagram actor Staff as Staff Trigger participant Gradio as Gradio UI participant FastAPI as FastAPI API participant VLM as VLM Client participant Semantic as Semantic Client Staff->>Gradio: Select Image Staff->>Gradio: Click Validate Gradio->>FastAPI: POST /validate FastAPI->>FastAPI: Preprocess image FastAPI->>VLM: analyze_plate() VLM->>VLM: OVMS: POST /v3/chat/completions VLM-->>FastAPI: Detected items FastAPI->>Semantic: match_items() Semantic-->>FastAPI: Similarity scores FastAPI-->>Gradio: Validation result Gradio-->>Staff: Display results ``` ### Docker Services | Container | Image | Ports | Description | | ------------------------- | ---------------------------------------- | ---------- | ----------------------------------- | | `dinein_app` | `intel/order-accuracy-dine-in:2026.3.0-rc1` | 7861, 8083 | Main application (Gradio + FastAPI) | | `dinein_ovms_vlm` | `openvino/model_server:2026.3.1-gpu` | 8002 | Vision-Language Model server | | `dinein_semantic_service` | `intel/semantic-search-agent:2026.3.0-rc1` | 8081, 9091 | Semantic text matching | | `metrics-collector` | `intel/hl-ai-metrics-collector:2026.1.0` | 9000 | System metrics aggregation | ### Network Topology ![Network Topology](./_assets/OA-DI-network-topology.svg "dine-in network topology") Host-exposed endpoints: ```text localhost:7861 ← Gradio UI localhost:8083 ← REST API localhost:8083/docs ← Swagger Docs localhost:8002 ← OVMS VLM localhost:9000 ← Metrics API localhost:8011/mcp ← MCP Server (events, read tools) ``` --- ## Component Details ### 1. VLM Client (`vlm_client.py`) The VLM Client handles communication with OpenVINO™ Model Server for visual inference. **Features:** - **Image Preprocessing**: Smart resizing (672px max), JPEG compression (82% quality), contrast enhancement - **Circuit Breaker**: 5 failures → OPEN, 30s recovery → HALF_OPEN, 2 successes → CLOSED - **Connection Pooling**: Shared `httpx.AsyncClient` with HTTP/2, 50 max connections - **Inventory-Aware Prompts**: Includes known menu items for improved accuracy ```python # Circuit Breaker States class CircuitState(Enum): CLOSED = "closed" # Normal operation OPEN = "open" # Failing, reject requests HALF_OPEN = "half_open" # Testing recovery ``` ### 2. Semantic Client (`semantic_client.py`) Handles fuzzy string matching for item comparison. **Features:** - **Similarity Threshold**: Default 0.7 (70% match required) - **Fallback Matching**: Exact string match when service unavailable - **Circuit Breaker**: 15s recovery timeout (faster than VLM) - **Connection Pool**: Shared client with 20 max connections ### 3. Validation Service (`validation_service.py`) Orchestrates the validation workflow using Strategy pattern. **Validation Pipeline:** 1. VLM inference → detected items 2. Semantic matching → item correlations 3. Quantity analysis → mismatches 4. Accuracy calculation → final score ```python # Accuracy Calculation accuracy = matched_items / max(expected_items, detected_items) order_complete = (missing == 0) and (quantity_errors == 0) and (extra == 0) ``` ### 4. Configuration Manager (`config.py`) Thread-safe singleton for application configuration. **Features:** - Double-checked locking pattern - Environment variable driven - Runtime benchmark mode toggle ### 5. API Layer (`api.py`) FastAPI endpoints with bounded validation cache. **Features:** - **BoundedValidationCache**: LRU eviction, 10K max entries - **Thread-safe service init**: Lock-protected lazy initialization - **Async metrics collection**: Non-blocking system stats ## Data Flow ### Validation Request Processing 1. **Image Processing**: Raw Image → Auto-Orient → Resize (672px) → Enhance → Sharpen → JPEG Compress (82%) → Base64 Encode 2. **VLM Inference**: Prompt: "Analyze this food plate image..." + Inventory list for context → OVMS POST `/v3/chat/completions` → Parse JSON response for detected items 3. **Semantic Matching**: For each expected item: - Find best match in detected items (similarity > 0.7) - Track: matched, missing, extra, quantity mismatches 4. **Result Aggregation**: ```text { "order_complete": true/false, "accuracy_score": 0.0-1.0, "missing_items": [...], "extra_items": [...], "metrics": { "latency": [...], "tps": [...], "utilization": [...] } } ``` ### Metrics Collection ![Metrics Pipeline](./_assets/OA-DI-metrics-pipeline.svg "metrics pipeline") ## Production Features ### Circuit Breaker Pattern Prevents cascading failures when external services are unhealthy. ```mermaid flowchart LR CLOSED["CLOSED"] OPEN["OPEN"] HALFOPEN["HALF-OPEN"] CLOSED -- "5 net failures (each success removes one failure)" --> OPEN OPEN -- "30s timeout" --> HALFOPEN HALFOPEN -- "2 successes" --> CLOSED HALFOPEN -- "1 failure" --> OPEN ``` ### Connection Pooling ```python # VLM Client Pool Configuration limits = httpx.Limits( max_keepalive_connections=20, max_connections=50, keepalive_expiry=30.0 ) timeout = httpx.Timeout( connect=10.0, read=300.0, # Extended for VLM inference write=10.0, pool=10.0 ) client = httpx.AsyncClient(limits=limits, timeout=timeout, http2=True) ``` ### Bounded Cache (LRU) ```python class BoundedValidationCache: """Thread-safe LRU cache with automatic eviction""" def __init__(self, maxsize: int = 10000): self._cache = OrderedDict() self._maxsize = maxsize self._lock = threading.Lock() def __setitem__(self, key, value): with self._lock: if key in self._cache: self._cache.move_to_end(key) self._cache[key] = value # Evict oldest when full while len(self._cache) > self._maxsize: self._cache.popitem(last=False) ``` --- ## MCP Server & Event Log (Agent Interface) Order Accuracy is a **sensor**, not an actor: it detects and reports order outcomes but never takes runtime actions. This is exposed to agents/other services through an [MCP](https://modelcontextprotocol.io) server built directly on FastMCP via a small local `mcp_sensor` module, separate from the REST API used by the Gradio UI. ```text Order Result (api.py) │ ▼ create_order_event() (src/services/order_events.py) │ ▼ emit_order_result() (src/mcp_service.py) │ ▼ Durable SQLite Event Log (results/order_accuracy_events.db) │ ▼ MCP Server (src/mcp_server.py, port 8011, path /mcp) │ ▼ MCP Client / Agent (read tools only, no action tools) ``` Every completed validation from `/api/validate` or `/api/validate/batch` emits an `order_validated` or `order_failed` event, which is durably logged **before** it is made available to any reader — this guarantees the log survives process restarts and that no event is lost. See the [Dine-In README](https://github.com/intel-retail/order-accuracy/blob/main/dine-in/README.md#mcp-server-events-durable-log-read-tools) for the full event schema, read-tool list, and a runnable client example. The separate benchmark/stream-density worker (`dinein-worker`) does not participate in this flow — only real validation traffic emits events. --- ## Performance Characteristics ### Latency Breakdown | Stage | Typical Duration | | ------------------- | ---------------- | | Image Preprocessing | 50–100 ms | | VLM Inference | 8–12 s | | Semantic Matching | 20–50 ms | | **Total E2E** | **9–15 s** | Target: < 15 s end-to-end for operational efficiency. ## System Requirements See the [System Requirements](./get-started/system-requirements.md) for detailed hardware, software, and network prerequisites.