API Reference#

The router exposes an OpenAI-compatible API. All examples assume the router is running on localhost:8000.

Service Info#

Endpoint:

GET /

Example:

curl http://localhost:8000/

Description:

Returns service name, version, status, and a map of available endpoints.

Response:

200 OK:

{
    "name": "Inference Router API",
    "version": "0.1.0",
    "status": "running",
    "endpoints": {
        "health": "/health",
        "chat": "/v1/chat/completions",
        "models": "/v1/models",
        "metrics": "/v1/metrics",
        "config": "/v1/config",
        "routing": "/v1/routing",
        "providers": "/v1/providers",
        "policies": "/v1/policies",
        "strategies": "/v1/strategies",
        "audio_transcriptions": "/v1/audio/transcriptions",
        "audio_speech": "/v1/audio/speech",
        "embeddings": "/v1/embeddings",
        "rerank": "/v1/rerank",
        "ocr": "/v1/ocr"
    }
}

Health Check#

Endpoint:

GET /health

Example:

curl http://localhost:8000/health

Description:

Verifies that the router is alive and operational. Includes router initialization status and current concurrency counters.

Response:

200 OK:

{
    "status": "healthy",
    "router": "initialized",
    "timestamp": 1733040000,
    "concurrency": {
        "active_requests": 0,
        "max_concurrency": 3
    }
}

max_concurrency is the integer limit, or the string "unlimited" when no limit is set.

503 Service Unavailable:
```
{"detail": "Router not initialized"}
```

Detailed Health Check#

Endpoint:

GET /health/detailed

Example:

curl http://localhost:8000/health/detailed

Description:

Runs a live health check against every provider and returns their individual status. Probes the backends, and therefore is more resource-intensive and takes longer to execute than the GET /health endpoint.

Response:

200 OK:
```
{
    "status": "healthy",
    "providers": {
        "local": {"status": "healthy"},
        "cloud": {"status": "healthy"}
    }
}
```
The keys in the providers map are the provider names configured in your setup. The value structure for each key depends on the specific provider’s health check structure.

List Models#

Endpoint:

GET /v1/models

Example:

curl http://localhost:8000/v1/models

Description:

Lists every available model. One entry per enabled provider in config.yaml, where id is the configured backend model name (the value clients pass in request.model to route here) and owned_by is the provider name. Two providers MAY share an id — they’re distinguishable by owned_by, and routing by model name picks the first such provider in configuration order; pass the provider name in request.model to target the other. The response always includes the virtual model "auto" for automatic routing.

Pass-through providers (type of transcription, tts, embeddings, rerank, or ocr) are not chat-capable models and are omitted from this list; they are reachable via their dedicated endpoints (see Pass-through Services) and manageable via /v1/providers.

Response:

200 OK:

{
    "object": "list",
    "data": [
        {
            "id": "Qwen/Qwen3-8B",
            "object": "model",
            "created": 1733040000,
            "owned_by": "local"
        },
        {
            "id": "MiniMax-M2.7",
            "object": "model",
            "created": 1733040000,
            "owned_by": "cloud"
        },
        {
            "id": "auto",
            "object": "model",
            "created": 1733040000,
            "owned_by": "inference-router"
        }
    ]
}

Chat Completions#

Endpoint:

POST /v1/chat/completions

Example:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello"}],
    "stream": false
  }'

Description:

OpenAI-compatible chat completion. Set model to a specific ID to pin the backend, or to "auto" for smart routing. Set stream: true for Server-Sent Events (SSE) streaming.

Request Body:

{
    "model": "auto",
    "messages": [
        {"role": "system", "content": "You are helpful."},
        {"role": "user", "content": "Hello"}
    ],
    "stream": false,
    "temperature": 0.7,
    "max_tokens": 200
}

model: Either "auto" for smart routing, a model ID from /v1/models (the primary path), or a configured provider name (legacy fallback — useful when two providers share a model ID and you need to target a specific one).
messages: List of OpenAI-format messages.
stream: When true, response is streamed as SSE.
Other OpenAI parameters, such as temperature, max_tokens, top_p, tools, tool_choice, and response_format, pass through to the backend.

Response (non-streaming):

200 OK:

{
    "id": "chatcmpl-...",
    "object": "chat.completion",
    "created": 1733040000,
    "model": "Qwen/Qwen3-8B",
    "choices": [
        {
            "index": 0,
            "message": {"role": "assistant", "content": "..."},
            "finish_reason": "stop"
        }
    ],
    "usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30}
}

Response (streaming):

200 OK with Content-Type: text/event-stream. Each chunk is an SSE data: {...} line. The stream ends with data: [DONE].

Errors:

400 Bad Request: routing failed (e.g. unknown model name).
4xx (forwarded): upstream client errors from the backend provider (e.g. an invalid payload) are forwarded verbatim with the provider’s own status code.
422 Unprocessable Entity: request validation failed.
429 Too Many Requests: concurrency limit reached.
500 Internal Server Error: inference or unexpected failure.
502 Bad Gateway: upstream provider error (non-4xx).
503 Service Unavailable: router not initialized.

Pass-through Services#

The router can expose additional OpenAI- and Cohere-compatible endpoints that forward the request body verbatim to a backing service and return the response untouched. Each is enabled by adding a provider whose type names the service (see Create or Update Provider and the Get Started) sections. If there is no such provider configured for a service, that endpoint returns 503.

Provider `type`	Endpoint	Body / Response
`transcription`	`POST /v1/audio/transcriptions`	multipart or JSON in, JSON out
`tts`	`POST /v1/audio/speech`	JSON in, binary audio out
`embeddings`	`POST /v1/embeddings`	JSON in, JSON out
`rerank`	`POST /v1/rerank`	JSON in, JSON out (Cohere-compatible)
`ocr`	`POST /v1/ocr`	multipart or JSON in, JSON out

Behavior (all pass-through endpoints):

The request body and Content-Type (including a multipart boundary) are forwarded unchanged; the client’s headers are passed through with Host stripped. Configured settings.auth on the provider is applied to the upstream call.
The upstream response status, body, and headers are returned as-is (hop-by-hop headers are regenerated).
Requests count against max-concurrency, shared with /v1/chat/completions.
These endpoints are managed dynamically through /v1/providers; enabling, disabling, or deleting the provider takes effect immediately.

Examples:

Transcribe audio:

curl http://localhost:8000/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=whisper-1"

Generate speech:

curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model": "tts-1", "voice": "alloy", "input": "Hello world"}' \
  -o speech.mp3

Create embeddings:

curl http://localhost:8000/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model": "bge-m3", "input": "hello world"}'

Rerank documents:

curl http://localhost:8000/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rerank-english-v3.0",
    "query": "What is the router?",
    "documents": ["A routing service", "A storage service"]
  }'

Run OCR:

curl http://localhost:8000/v1/ocr \
  -H "Content-Type: application/json" \
  -d '{"image_path": "/data/page.png"}'

Errors:

429 Too Many Requests: concurrency limit reached.
502 Bad Gateway: could not reach the backing service.
503 Service Unavailable: no provider configured for this service, or router not initialized.
504 Gateway Timeout: the backing service did not respond within settings.timeout.

Metrics#

Endpoint:

GET /v1/metrics

Example:

curl http://localhost:8000/v1/metrics

Description:

Aggregated routing, token, and latency metrics, bucketed by provider name. Counters accumulate from process start (or the last POST /v1/metrics/reset) across both streaming and non-streaming requests.

Response:

200 OK: object with three top-level sections:

The by_provider map keys are "<provider>/<model>", where each key is the configured provider name followed by the backend model ID; for example, "local/Qwen/Qwen3.5-9B". When one provider serves multiple models, or two providers expose the same model, each (provider, model) pair gets its own bucket so dashboards can disambiguate them. The field name remains by_provider for back-compat with pre-existing dashboards; only the key strings changed.

routing_stats — total request count and per-bucket request counts.
token_metrics — per-bucket input, output, and total token counts plus request_share and token_share (fractions of the overall traffic), and an overall aggregate. Also carries two request-level token breakdowns: before_router (the raw request) and after_router (the request actually forwarded to the backend after router plugins run). Both use the same tiktoken unit, so the delta between them is the token saving from router-side processing (e.g. context compression). Each breakdown reports system_prompt_tokens, tool_schema_tokens, context_tokens, and overall_tokens.
latency_metrics — per-bucket average end-to-end latency, TTFT (time-to-first-token), and TPOT (time-per-output-token), plus an overall aggregate. TTFT is reported only for streaming requests; non-streaming requests contribute to avg_latency_ms only.

Example:

{
    "routing_stats": {
        "total_requests": 12,
        "by_provider": {
            "local/Qwen/Qwen3.5-9B": 8,
            "cloud/MiniMax-M2.7": 4
        }
    },
    "token_metrics": {
        "by_provider": {
            "local/Qwen/Qwen3.5-9B": {
                "input_tokens": 1200,
                "output_tokens": 800,
                "total_tokens": 2000,
                "request_count": 8,
                "avg_tokens_per_request": 250.0,
                "request_share": 0.667,
                "token_share": 0.625
            }
        },
        "overall": {
            "total_tokens": 3200,
            "total_input_tokens": 1900,
            "total_output_tokens": 1300,
            "total_requests": 12,
            "avg_tokens_per_request": 266.7
        },
        "before_router": {
            "system_prompt_tokens": 300,
            "tool_schema_tokens": 150,
            "context_tokens": 1450,
            "overall_tokens": 1900
        },
        "after_router": {
            "system_prompt_tokens": 300,
            "tool_schema_tokens": 150,
            "context_tokens": 1100,
            "overall_tokens": 1550
        }
    },
    "latency_metrics": {
        "by_provider": {
            "local/Qwen/Qwen3.5-9B": {
                "avg_latency_ms": 420.15,
                "avg_ttft_ms": 35.20,
                "avg_tpot_ms": 4.8123,
                "ttft_count": 5,
                "tpot_count": 5
            }
        },
        "overall": {
            "avg_latency_ms": 510.40,
            "avg_ttft_ms": 38.10,
            "avg_tpot_ms": 5.1042,
            "ttft_count": 7,
            "tpot_count": 7
        }
    }
}

503 Service Unavailable:

{"detail": "Telemetry not initialized"}

Reset Metrics#

Endpoint:

POST /v1/metrics/reset

Example:

curl -X POST http://localhost:8000/v1/metrics/reset

Description:

Clears all telemetry metrics.

Response:

200 OK:

{
    "status": "success",
    "message": "All statistics metrics have been reset",
    "timestamp": 1733040000
}

Get Configuration#

Endpoint:

GET /v1/config

Example:

curl http://localhost:8000/v1/config

Description:

Returns the current in-memory router configuration. Sensitive values (keys named api_key, token, secret, or password) are replaced with "***REDACTED***" in the response.

Response:

200 OK:

{
    "object": "config",
    "data": {
        "log_level": "INFO",
        "providers": [
            {
                "name": "local",
                "type": "openai",
                "model": "Qwen/Qwen3-8B",
                "enabled": true,
                "metadata": {},
                "settings": {"api_key": "***REDACTED***"}
            }
        ],
        "plugins": {
            "prerouting": [],
            "postrouting": [],
            "postresponse": []
        },
        "routing": {"policy": "default", "strategy": "auto"},
        "telemetry": {
            "backend": "memory",
            "enabled": true,
            "file_path": null
        },
        "cors_origins": ["*"]
    },
    "path": "/path/to/config.yaml",
    "warnings": []
}

path is the on-disk configuration file path (or null if not configured). warnings lists non-fatal advisories about the configuration.

503 Service Unavailable:
```
{"detail": "Router not initialized"}
```

Get Routing#

Endpoint:

GET /v1/routing

Example:

curl http://localhost:8000/v1/routing

Description:

Returns the active routing policy — the policy field of the configuration’s routing section. The policy names an entry in policy.yaml that decides how requests are dispatched across providers.

Response:

200 OK:
```
{"policy": "Balanced"}
```
policy is null if none is configured (the router then falls back to the first policy defined in policy.yaml).
503 Service Unavailable:
```
{"detail": "Router not initialized"}
```

Update Routing#

Endpoint:

POST /v1/routing

Example:

curl http://localhost:8000/v1/routing \
  -H "Content-Type: application/json" \
  -d '{"policy": "Balanced"}'

Description:

Sets the routing policy, then persists the change to the on-disk configuration and rebuilds the runtime. The policy must name an entry in policy.yaml; an unknown name is rejected and the previous policy stays in effect. Provider secrets in config.yaml (e.g. ${VAR} placeholders) are left untouched.

Request Body:

{"policy": "Balanced"}

policy: the routing policy name (must exist in policy.yaml).

Response:

200 OK: the resulting routing policy.
```
{"policy": "Balanced"}
```
400 Bad Request: the rebuilt runtime failed to initialize — e.g. the policy name is not found in policy.yaml.
500 Internal Server Error: failed to persist the configuration.
503 Service Unavailable: router not initialized.

List Providers#

Endpoint:

GET /v1/providers

Example:

curl http://localhost:8000/v1/providers

Description:

Lists all configured providers. Sensitive values (keys named api_key, token, secret, or password) are replaced with "***REDACTED***" in the response.

Response:

200 OK:

{
    "object": "list",
    "data": [
        {
            "name": "local",
            "type": "hosted_vllm",
            "model": "Qwen/Qwen3.5-9B",
            "enabled": true,
            "metadata": {
                "labels": ["planning", "local"],
                "cost": 0,
                "performance": 0.85,
                "capability": {"complexity": 0.75}
            },
            "settings": {
                "endpoint": "http://localhost:5000/v1",
                "timeout": 300.0,
                "auth": {"scheme": "none", "api_key": null, "custom_headers": {}}
            }
        }
    ]
}

503 Service Unavailable: router not initialized.

Get Provider#

Endpoint:

GET /v1/providers/{name}

Example:

curl http://localhost:8000/v1/providers/local

Description:

Returns a single provider’s configuration, identified by its unique name. Secrets are redacted as in GET /v1/providers.

Response:

200 OK: a single ProviderResponse (same structure as an entry in the list above).

404 Not Found: no provider with that name.

{"detail": "Provider 'local' not found"}

Create or Update Provider#

Endpoint:

POST /v1/providers/{name}

Example:

curl http://localhost:8000/v1/providers/openai \
  -H "Content-Type: application/json" \
  -d '{
    "type": "openai",
    "model": "gpt-4o",
    "enabled": true,
    "settings": {
      "endpoint": "https://api.openai.com/v1",
      "auth": {"scheme": "bearer", "api_key": "${OPENAI_API_KEY}"}
    }
  }'

Description:

Creates or updates a provider, then persists the change to the on-disk configuration and rebuilds the runtime. The provider is created if it does not exist, and updated otherwise. Only the fields present in the request body are changed; settings and metadata, when supplied, replace the existing section wholesale.

Secrets are preserved on disk: an api_key written as an environment placeholder (e.g. "${OPENAI_API_KEY}") is stored verbatim in config.yaml rather than being resolved to its value. Providers that are left untouched keep their existing placeholders unchanged.

Request Body:

{
    "type": "openai",
    "model": "gpt-4o",
    "enabled": true,
    "metadata": {"labels": ["cloud"], "cost": 5},
    "settings": {
        "endpoint": "https://api.openai.com/v1",
        "auth": {"scheme": "bearer", "api_key": "${OPENAI_API_KEY}"}
    }
}

type: provider type (e.g. hosted_vllm, openai). Required when creating a new provider; optional on update.
model: backend model identifier. Required when creating; optional on update.
enabled: optional. Toggle the provider on or off.
metadata: optional. Routing metadata (labels, cost, performance, capability; extra fields allowed).
settings: optional. Provider-specific settings such as endpoint, timeout, and auth (extra fields allowed).

Response:

200 OK: the resulting ProviderResponse (secrets redacted).
400 Bad Request: creating a new provider without both type and model, or the rebuilt runtime failed to initialize (e.g. no enabled providers remain).
500 Internal Server Error: failed to persist the configuration.
503 Service Unavailable: router not initialized.

Delete Provider#

Endpoint:

DELETE /v1/providers/{name}

Example:

curl -X DELETE http://localhost:8000/v1/providers/openai

Description:

Removes a provider, persists the change to the on-disk configuration, and rebuilds the runtime.

Response:

200 OK:

{"status": "success", "message": "Provider 'openai' deleted"}

400 Bad Request: the rebuilt runtime failed to initialize (e.g. deleting the last enabled provider). The change is rejected atomically — the configuration file is left unchanged.
404 Not Found: no provider with that name.
500 Internal Server Error: failed to delete the provider.
503 Service Unavailable: router not initialized.

List Plugins#

Endpoint:

GET /v1/plugins

Example:

curl http://localhost:8000/v1/plugins

Description:

Lists all configured plugins.

Response:

200 OK:

{
    "object": "list",
    "data": [
        {
            "name": "dummy",
            "node": "dummy_logger",
            "enabled": true,
            "trigger": "prerouting",
            "settings": {}
        }
    ]
}

503 Service Unavailable: router not initialized.

Terminology: a node is a plugin type (a registered plugin class, e.g. dummy_logger); a name is one configured instance of that type. Instance routes are ordered node-first: /v1/plugins/{node}/{name}.

List Plugin Nodes#

Endpoint:

GET /v1/plugins/nodes

Example:

curl http://localhost:8000/v1/plugins/nodes

Description:

Lists the plugin types (nodes) registered in code, independent of what is configured. Each entry carries the type’s metadata and its settings JSON schema, so a client can discover what a node accepts before configuring an instance.

Response:

200 OK:

{
    "object": "list",
    "data": [
        {
            "node": "dummy_logger",
            "plugin_group": "",
            "description": "Prints which phase invoked it; ...",
            "settings_schema": {"type": "object", "properties": {"label": {"type": "string"}}}
        }
    ]
}

Get Plugin Node#

Endpoint:

GET /v1/plugins/{node}

Example:

curl http://localhost:8000/v1/plugins/dummy_logger

Description:

Node-level view of a plugin type, as defined by that plugin class’s describe_node() hook. The default payload is the type metadata (as in GET /v1/plugins/nodes); a plugin may override describe_node() to add node-level aggregate info (e.g. metrics spanning all instances of the type).

Response:

200 OK: a plugin-defined object (default = the node’s metadata).

404 Not Found: {node} is not a registered plugin type.

{"detail": "Plugin node 'dummy_logger' not registered"}

Get Plugin#

Endpoint:

GET /v1/plugins/{node}/{name}

Example:

curl http://localhost:8000/v1/plugins/dummy_logger/dummy

Description:

Instance-level view of a configured plugin, as defined by the plugin’s describe() hook. Prefers the live instance (so per-instance runtime info such as metrics is folded in); falls back to the static configuration view for a configured-but-disabled plugin that is not loaded.

Response:

200 OK (a live dummy_logger instance folds its metrics into describe()):

{
    "name": "dummy",
    "node": "dummy_logger",
    "trigger": "prerouting",
    "enabled": true,
    "settings": {"label": "demo"},
    "metrics": {"process_request": 12, "process_response": 0}
}

404 Not Found: no plugin with that node/name.

{"detail": "Plugin 'dummy' with node 'dummy_logger' not found"}

Create or Update Plugin#

Endpoint:

POST /v1/plugins/{node}/{name}

Example:

curl http://localhost:8000/v1/plugins/dummy_logger/dummy \
  -H "Content-Type: application/json" \
  -d '{
    "enabled": true,
    "trigger": "prerouting",
    "settings": {"label": "demo"}
  }'

Description:

Creates or updates a plugin instance, then persists the change to the on-disk configuration and rebuilds the runtime. The instance is created if it does not exist, and updated otherwise. Only the fields present in the request body are changed.

Request Body:

{
    "enabled": true,
    "trigger": "prerouting",
    "settings": {"label": "demo"}
}

enabled: optional. Toggle the plugin on or off.
trigger: optional. One of prerouting, postrouting, or postresponse.
settings: optional. Plugin-specific settings object (extra fields allowed).

Response:

200 OK: the resulting PluginResponse (name, node, enabled, trigger, settings).
500 Internal Server Error: failed to persist the configuration.
503 Service Unavailable: router not initialized.

Delete Plugin#

Endpoint:

DELETE /v1/plugins/{node}/{name}

Example:

curl -X DELETE http://localhost:8000/v1/plugins/dummy_logger/dummy

Description:

Removes a plugin instance, persists the change to the on-disk configuration, and rebuilds the runtime.

Response:

200 OK:

{"status": "success", "message": "Plugin 'dummy' deleted"}

404 Not Found: no plugin with that node/name.
500 Internal Server Error: failed to delete the plugin.
503 Service Unavailable: router not initialized.

Reset Plugin Instance#

Endpoint:

POST /v1/plugins/{node}/{name}/reset

Example:

curl -X POST http://localhost:8000/v1/plugins/dummy_logger/dummy/reset

Description:

Resets one plugin instance’s own runtime state (e.g. per-instance metrics) via its reset() hook. Acts on the live plugin, so the instance must be loaded (enabled). Applies only to plugins that support reset; a plugin that does not, returns 400.

Response:

200 OK:

{"status": "success", "message": "Reset plugin 'dummy'"}

400 Bad Request: the plugin does not support reset.

{"detail": "Plugin 'dummy' with node 'dummy_logger' does not support reset"}

404 Not Found: the plugin is not loaded.
503 Service Unavailable: router not initialized.

Reset Plugin Node#

Endpoint:

POST /v1/plugins/{node}/reset

Example:

curl -X POST http://localhost:8000/v1/plugins/dummy_logger/reset

Description:

Resets node-level (type/group-wide) state for a plugin type via its reset_node() classmethod hook. Applies only to types that support it; a type that does not, returns 400.

Response:

200 OK:

{"status": "success", "message": "Reset plugin node 'dummy_logger'"}

400 Bad Request: the node does not support reset.
404 Not Found: {node} is not a registered plugin type.

Policies#

A policy is a named, ordered list of strategies plus a criterion that decides how their results are combined during routing. Policies are stored in policy.yaml; each policy is addressed by its unique name.

All policy endpoints target <workspace>/policy.yaml, which must already exist: the API reads and edits an operator’s workspace copy and never mutates the bundled defaults in the source tree. Any request against a missing workspace file returns 400. Mutations are validated with the same rules applied at startup and, on success, written atomically and applied immediately.

A policy object has the following structure:

{
    "name": "Balanced",
    "criterion": "FirstMatch",
    "strategies": ["Planning", "ContextLengthQuality"]
}

name: unique identifier. Restricted to letters, digits, ., -, and _.
criterion: how strategy results are combined. One of FirstMatch (default) or AllMatch.
strategies: non-empty, ordered list of strategy names. Each must name a strategy that exists in strategy.yaml.

List Policies#

Endpoint:

GET /v1/policies

Example:

curl http://localhost:8000/v1/policies

Description:

Lists all policies defined in policy.yaml.

Response:

200 OK:

[
    {
        "name": "Balanced",
        "criterion": "FirstMatch",
        "strategies": ["Planning", "ContextLengthQuality"]
    },
    {
        "name": "CostFirst",
        "criterion": "FirstMatch",
        "strategies": ["ZeroCost"]
    }
]

400 Bad Request: the workspace policy.yaml does not exist.
500 Internal Server Error: policy.yaml is invalid.

Get Policy#

Endpoint:

GET /v1/policies/{name}

Example:

curl http://localhost:8000/v1/policies/Balanced

Description:

Returns a single policy identified by its unique name.

Response:

200 OK: a single policy object (same structure as an entry in the list above).

404 Not Found: no policy with that name.

{"detail": "Policy 'Balanced' not found"}

Create or Update Policy#

Endpoint:

POST /v1/policies/{name}

Example:

curl http://localhost:8000/v1/policies/Balanced \
  -H "Content-Type: application/json" \
  -d '{
    "criterion": "FirstMatch",
    "strategies": ["Planning", "ContextLengthQuality"]
  }'

Description:

Creates or updates a policy, then persists the change to policy.yaml. The policy is created if it does not exist, and replaced wholesale otherwise. The name is taken from the path; a name in the body is ignored. A GET response round-trips as a POST payload.

Request Body:

{
    "criterion": "FirstMatch",
    "strategies": ["Planning", "ContextLengthQuality"]
}

criterion: optional. FirstMatch (default) or AllMatch.
strategies: required. Non-empty, ordered list of strategy names, each of which must exist in strategy.yaml.

Response:

200 OK: the resulting policy object.
400 Bad Request: the body is invalid — e.g. missing or empty strategies, an invalid criterion, an invalid name, or a strategy that does not exist in strategy.yaml; or the workspace policy.yaml does not exist.
```
{"detail": "Policy 'Balanced' references unknown strategy 'Planing'"}
```
500 Internal Server Error: failed to persist policy.yaml.

Delete Policy#

Endpoint:

DELETE /v1/policies/{name}

Example:

curl -X DELETE http://localhost:8000/v1/policies/CostFirst

Description:

Removes a policy and persists the change to policy.yaml.

Response:

200 OK:

{"status": "success", "message": "Policy 'CostFirst' deleted"}

400 Bad Request: the workspace policy.yaml does not exist.
404 Not Found: no policy with that name.
409 Conflict: the policy is the router’s active routing policy (routing.policy) and cannot be removed while in use.
```
{"detail": "Policy 'Balanced' is the active routing policy and cannot be deleted"}
```
500 Internal Server Error: failed to persist policy.yaml.

Strategies#

A strategy is a named rule set plus a provider selector: its rules decide whether the strategy matches a request, and its provider_selector (and optional sort) pick and rank the providers that can serve it. Strategies are stored in strategy.yaml, and policies reference them by name.

All strategy endpoints target <workspace>/strategy.yaml, which must already exist: the API reads and edits an operator’s workspace copy and never mutates the bundled defaults in the source tree. Any request against a missing workspace file returns 400. Mutations are validated with the same rules applied at startup and, on success, written atomically and applied immediately.

A strategy object has the following structure:

{
    "name": "Planning",
    "description": "Routes based on message content indicating planning intent.",
    "rules": [
        {"type": "MessageContentRule", "param": {"pattern": "plan", "roles": ["user"]}}
    ],
    "provider_selector": {
        "label": "planning",
        "capability": {"complexity": 0.7}
    },
    "sort": [],
    "require_healthy": false,
    "limit": null
}

name: unique identifier. Restricted to letters, digits, ., -, and _.
description: optional free-text description.
rules: optional list of {type, param} rule entries. type must be a built-in rule (see Built-in Rules); param holds that rule’s constructor arguments. A strategy with no rules always matches.
provider_selector: required. Criteria for selecting providers — label, cost, and capability (complexity, tool_calling). Scalars apply unconditionally; mappings are keyed by the zone index produced by a zone rule.
sort: optional list of {provider_attribute, descending} entries that rank matched providers.
require_healthy: optional. When true, only providers passing a live health check are considered.
limit: optional integer cap on the number of ranked candidates returned.

List Strategies#

Endpoint:

GET /v1/strategies

Example:

curl http://localhost:8000/v1/strategies

Description:

Lists all strategies defined in strategy.yaml.

Response:

200 OK:

[
    {
        "name": "Planning",
        "description": "Routes based on message content indicating planning intent.",
        "rules": [
            {"type": "MessageContentRule", "param": {"pattern": "plan", "roles": ["user"]}}
        ],
        "provider_selector": {"label": "planning", "capability": {"complexity": 0.7}},
        "sort": [],
        "require_healthy": false,
        "limit": null
    }
]

400 Bad Request: the workspace strategy.yaml does not exist.
500 Internal Server Error: strategy.yaml is invalid.

Get Strategy#

Endpoint:

GET /v1/strategies/{name}

Example:

curl http://localhost:8000/v1/strategies/Planning

Description:

Returns a single strategy, identified by its unique name.

Response:

200 OK: a single strategy object (same structure as an entry in the list above).

404 Not Found: no strategy with that name.

{"detail": "Strategy 'Planning' not found"}

Create or Update Strategy#

Endpoint:

POST /v1/strategies/{name}

Example:

curl http://localhost:8000/v1/strategies/Planning \
  -H "Content-Type: application/json" \
  -d '{
    "description": "Routes based on message content indicating planning intent.",
    "rules": [
      {"type": "MessageContentRule", "param": {"pattern": "plan", "roles": ["user"]}}
    ],
    "provider_selector": {"label": "planning", "capability": {"complexity": 0.7}}
  }'

Description:

Creates or updates a strategy, then persists the change to strategy.yaml. The strategy is created if it does not exist, and replaced wholesale otherwise. The name is taken from the path; a name in the body is ignored. A GET response round-trips as a POST payload.

Request Body:

{
    "description": "Routes based on message content indicating planning intent.",
    "rules": [
        {"type": "MessageContentRule", "param": {"pattern": "plan", "roles": ["user"]}}
    ],
    "provider_selector": {"label": "planning", "capability": {"complexity": 0.7}},
    "sort": [],
    "require_healthy": false,
    "limit": null
}

provider_selector: required. See the object structure above.
description, rules, sort, require_healthy, limit: optional.

Response:

200 OK: the resulting strategy object.
400 Bad Request: the body is invalid — e.g. a missing provider_selector, an unknown rule type, invalid rule param, or an invalid name; or the workspace strategy.yaml does not exist.
```
{"detail": "Unknown rule class 'MessageContntRule'"}
```
500 Internal Server Error: failed to persist strategy.yaml.

Delete Strategy#

Endpoint:

DELETE /v1/strategies/{name}

Example:

curl -X DELETE http://localhost:8000/v1/strategies/Planning

Description:

Removes a strategy and persists the change to strategy.yaml. A strategy that is still referenced by a policy cannot be deleted, to avoid leaving a policy pointing at a missing strategy. Delete or update the referencing policy first, then delete the strategy.

Response:

200 OK:

{"status": "success", "message": "Strategy 'Planning' deleted"}

400 Bad Request: the workspace strategy.yaml does not exist.
404 Not Found: no strategy with that name.

409 Conflict: the strategy is still referenced by one or more policies.

{"detail": "Strategy 'Planning' is referenced by policies: Balanced"}

500 Internal Server Error: failed to persist strategy.yaml.

API Reference#

Service Info#

Health Check#

Detailed Health Check#

List Models#

Chat Completions#

Pass-through Services#

Metrics#

Reset Metrics#

Get Configuration#

Get Routing#

Update Routing#

List Providers#

Get Provider#

Create or Update Provider#

Delete Provider#

List Plugins#

List Plugin Nodes#

Get Plugin Node#

Get Plugin#

Create or Update Plugin#

Delete Plugin#

Reset Plugin Instance#

Reset Plugin Node#

Policies#

List Policies#

Get Policy#

Create or Update Policy#

Delete Policy#

Strategies#

List Strategies#

Get Strategy#

Create or Update Strategy#

Delete Strategy#

This Page