How It Works#

This section describes the architecture and internal request flow for comparison requests in the Semantic Search Agent microservice.

Architecture#

At a high level, the Semantic Search Agent accepts item comparison payloads via REST, passes them through a configured matching strategy, and returns structured results. The matching pipeline uses a two-pass approach to minimize latency and inference costs: fast exact normalization first, followed by VLM-based semantic reasoning for any remaining unmatched items.

        %%{init: {
  'theme': 'base',
  'themeVariables': {
    'fontFamily': '"IntelOne Display", "Intel Clear", "Inter", "Segoe UI", Arial, sans-serif',
    'fontSize': '14px',
    'primaryColor': '#0068B5',
    'primaryTextColor': '#FFFFFF',
    'primaryBorderColor': '#00377C',
    'lineColor': '#00377C',
    'secondaryColor': '#EEF3F8',
    'tertiaryColor': '#F7F8FA',
    'background': '#FFFFFF',
    'mainBkg': '#FFFFFF',
    'clusterBkg': '#F7F8FA',
    'clusterBorder': '#0068B5',
    'edgeLabelBackground': '#FFFFFF',
    'noteBkgColor': '#F7F8FA',
    'noteTextColor': '#3A3A3A'
  }
}}%%
flowchart TD
    Client([Client])

    subgraph Service["Semantic Search Agent (:8080)"]
        API["API Layer<br/>(REST Endpoints)"]

        subgraph Engine["Comparison Engine"]
            ExactPass["Pass 1: Exact Matcher<br/>(Normalize → Compare)"]
            SemanticPass["Pass 2: Semantic Matcher<br/>(VLM Prompt → YES/NO)"]
            Cache[("Result Cache<br/>(Memory / Redis)")]
        end

        subgraph VLM["VLM Backend"]
            Factory["VLMBackendFactory"]
            OVMS["OVMS Backend<br/>(OpenAI-compat API)"]
            OVLocal["OpenVINO Local Backend<br/>(In-process GenAI)"]
            OAI["OpenAI Backend<br/>(Cloud API)"]
        end

        Config["Settings & Config Files<br/>(inventory.json / orders.json)"]
        Metrics["Prometheus Metrics<br/>(:9090/metrics)"]
    end

    Client -- "POST /api/v1/compare/order" --> API
    Client -- "POST /api/v1/compare/inventory" --> API
    Client -- "POST /api/v1/compare/semantic" --> API
    API --> ExactPass
    ExactPass -->|Unmatched items| SemanticPass
    SemanticPass <--> Cache
    SemanticPass --> Factory
    Factory --> OVMS
    Factory --> OVLocal
    Factory --> OAI
    API <--> Config
    API --> Metrics

    classDef client fill:#FFFFFF,stroke:#0068B5,stroke-width:2px,color:#3A3A3A;
    classDef core fill:#0068B5,stroke:#00377C,stroke-width:1.5px,color:#FFFFFF;
    classDef backend fill:#00A3F4,stroke:#00377C,stroke-width:1.5px,color:#FFFFFF;
    classDef store fill:#6C6C6C,stroke:#0068B5,stroke-width:1.5px,color:#FFFFFF;

    class Client client;
    class API,ExactPass,SemanticPass,Factory core;
    class OVMS,OVLocal,OAI backend;
    class Cache,Config,Metrics store;

    style Service fill:#F7F8FA,stroke:#0068B5,stroke-width:1.5px,color:#3A3A3A;
    style Engine fill:#EEF3F8,stroke:#0068B5,stroke-width:1.0px,color:#3A3A3A;
    style VLM fill:#EEF3F8,stroke:#0068B5,stroke-width:1.0px,color:#3A3A3A;

Key components:

  • API Layer — Accepts and validates incoming comparison requests using Pydantic models. Routes to the appropriate ComparisonEngine method and returns structured JSON responses.

  • ComparisonEngine — Orchestrates the two-pass matching pipeline. Loads order and inventory data from config JSON files. Coordinates exact and semantic matchers, aggregates results (missing, extra, quantity mismatch, and matched), and records Prometheus metrics.

  • ExactMatcher — Normalizes both input strings (lowercase, whitespace trimming, and special character removal) and performs direct string equality. Returns confidence 1.0 on match, 0.0 otherwise.

  • SemanticMatcher — Constructs a structured prompt from the input pair and a context string, submits it to the configured VLM backend, and interprets the YES/NO response as a boolean match. Checks an in-memory or Redis cache before invoking the VLM to avoid redundant inference calls.

  • HybridMatcher — Runs ExactMatcher first as a fast path. If the exact confidence meets the configured threshold (default 0.9), returns the exact result immediately. Otherwise, delegates to SemanticMatcher and returns the semantic result.

  • VLMBackendFactory — Singleton factory that creates and caches one VLM backend instance per backend type. Supports ovms, openvino_local, and openai backends.

  • OVMS Backend — Sends requests to an OpenVINO model server using the OpenAI-compatible /v3/chat/completions endpoint. Bypasses system proxy to communicate with internal OpenVINO model server hosts.

  • OpenVINO Local Backend — Loads an OpenVINO IR model in-process using the openvino-genai library. Suitable for GPU-accelerated edge deployments without a separate model server.

  • OpenAI Backend — Delegates to the OpenAI API for cloud-based inference. Used as a development or fallback option.

  • Cache — Keyed by Message Digest Algorithm 5 (MD5) hash of the normalized input pair and context string. Supports configurable time-to-live (TTL). Backed by either an in-process MemoryCache or an external RedisCache.

Request Flow#

Order Validation (POST /api/v1/compare/order)#

  1. Validate — The FastAPI framework validates the request body against OrderValidationRequest. Each item must have a name (string) and quantity (integer ≥ 1).

  2. Pass 1 — Exact Matching — For every expected item, the engine normalizes its name and searches detected items for an exact normalized match. On a match, the item is added to matched and the detected slot is reserved. If quantities differ, the item is added to quantity_mismatch.

  3. Pass 2 — Semantic Matching — For each expected item still unmatched after Pass 1, the engine iterates over unreserved detected items and calls matcher.match(expected_name, detected_name). If MatchResult.match is True, the item pair is added to matched with the semantic confidence. Unmatched expected items become missing; unreserved detected items become extra.

  4. Respond — Returns an OrderValidationResponse containing status (validated or mismatch), a full validation breakdown, and metrics (exact and semantic match counts, processing time, total expected items, and total detected items).

Inventory Validation (POST /api/v1/compare/inventory)#

  1. Validate — The FastAPI framework validates the request body against InventoryValidationRequest. Accepts a list of item name strings and an optional inventory list (uses config/inventory.json if omitted).

  2. Per-Item Matching — For each input item, exact match is attempted against all inventory entries. If no exact match is found and semantic matching is enabled, the engine iterates inventory entries and picks the highest-confidence semantic match above the threshold.

  3. Respond — Returns an InventoryValidationResponse containing per-item results (matched inventory item, match type, and confidence) and a summary (total items, matched, unmatched, and processing time in ms).

Semantic Match (POST /api/v1/compare/semantic)#

  1. Validate — The FastAPI framework validates the request body against SemanticMatchRequest with text1, text2, and an optional context string.

  2. Match — Directly calls SemanticMatcher.match(), which checks the cache first, then invokes the VLM backend with a structured prompt.

  3. Respond — Returns a SemanticMatchResponse containing the match (boolean) result, confidence (float) score, reasoning (VLM response), and match_type.

Configuration Surface#

All runtime settings are parsed and validated via Pydantic Settings on startup. Environment variables or a .env file at the project root override defaults. See the Configuration Guide for a comprehensive list of parameters.