Environment Variables#

Configuration is managed via environment variables. All variables map directly to Pydantic Settings field names (case-insensitive). The .env file is loaded automatically by docker compose up.

Core Settings#

Variable

Default

Description

HOST

0.0.0.0

API server bind address

METRICS_PORT

9090

Metrics Manager API port

SERVICE_NAME

metrics-manager

Service name used in logs and health checks

SERVICE_VERSION

2026.3.0

Service version reported in health endpoints

ENVIRONMENT

production

Deployment environment: development, staging, production. Production disables /docs and /redoc Swagger endpoints

LOG_LEVEL

INFO

Logging level: DEBUG, INFO, WARNING, ERROR, CRITICAL

LOG_FORMAT

json

Log format: json (structured, for log aggregators) or text (human-readable)

LOG_INCLUDE_TIMESTAMP

true

Include timestamp field in log entries

CORS_ORIGINS

*

Allowed CORS origins (comma-separated or JSON array of strings). Malformed JSON arrays or non-string entries fail startup. Set to http://localhost:3000,http://my-dashboard:3000 to restrict

CORS_ALLOW_CREDENTIALS

false

Allow credentials in CORS requests (cookies, etc.)

METRICS_MANAGER_HOSTNAME

(unset)

Override the host= tag stamped on every metric (Telegraf and the mm-plugin-* collectors). Unset = use kernel hostname. Set to a stable value (e.g., lab-node-42) to keep Grafana dashboards stable across reboots

Metrics Storage#

Variable

Default

Description

METRICS_RETENTION_SECONDS

300

How long to keep metrics in memory (seconds). After this duration, metrics are expired and removed on next store access

MAX_METRICS_BATCH_SIZE

1000

Maximum metrics per single batch request (POST /api/v1/metrics)

MAX_METRICS_IN_MEMORY

100000

Maximum metrics in memory. When exceeded, oldest entries are evicted automatically

CUSTOM_METRICS_DIR

/app/custom-metrics

Directory where custom metric scripts are executed (via Telegraf inputs.exec). Do not change inside container

FILE_PERSIST_DEBOUNCE_MS

100

Debounce interval for Telegraf HTTP persistence (milliseconds). Higher values reduce HTTP calls but increase latency. Range: 10–5000 ms

Rate Limiting#

Variable

Default

Description

RATE_LIMIT_ENABLED

true

Enable rate limiting by client IP

RATE_LIMIT_REQUESTS_PER_MINUTE

1000

Maximum requests per minute per client IP

RATE_LIMIT_BURST

100

Burst allowance (tokens available before rate limit kicks in)

Exempt paths: /health, /api/v1/stats, and SSE endpoints are NOT rate limited.

Performance#

Variable

Default

Description

ENABLE_GZIP_COMPRESSION

true

Enable gzip compression for HTTP responses >1 KB. Reduces bandwidth but increases CPU usage

Security#

Variable

Default

Description

TRUST_FORWARDED_HEADERS

false

Honor X-Forwarded-For / X-Real-IP headers for client IP detection. Set to true ONLY when running behind a trusted reverse proxy (Nginx, Traefik, etc.)

Warning: Setting this to true without a reverse proxy allows clients to spoof their IP and bypass rate limiting.

Telegraf Settings (Application Configuration)#

These variables configure the Telegraf locations for the application to use:

Variable

Default

Description

TELEGRAF_CONFIG_PATH

/etc/telegraf/telegraf.conf

Main Telegraf config inside the container, read by the entrypoint and passed to Telegraf

TELEGRAF_PORT

9273

Telegraf Prometheus endpoint port (where SSE poller fetches system metrics)

TELEGRAF_HTTP_PORT

8186

Docker-only Telegraf HTTP listener port; used by the shipped configuration

TELEGRAF_HTTP_ENDPOINT

http://localhost:8186/write

Telegraf HTTP listener endpoint used to persist custom metrics in InfluxDB Line Protocol. Must be accessible from the app container

SSE Poller Settings#

The SSE endpoint (/metrics/stream) polls the Telegraf Prometheus endpoint for each connected client independently. There is no shared queue.

Variable

Default

Description

PROMETHEUS_POLLER_INTERVAL_MS

500

Polling interval in milliseconds (100–5000). Lower values = more frequent SSE events but higher CPU usage per client. Recommended: 500 ms

PROMETHEUS_TELEGRAF_ENDPOINT

http://localhost:9273

Telegraf Prometheus endpoint polled by SSE clients. Must be accessible from the app container

Docker Compose Variables#

These variables are used ONLY by compose.yaml and are NOT read by the application:

Variable

Default

Description

TELEGRAF_CONFIG

./telegraf.conf

Host path to Telegraf config file (mounted into container)

TELEGRAF_CONFIG_DIR

./telegraf.d

Host directory mounted at /etc/telegraf/telegraf.d; .conf files are snapshotted and loaded at startup

HOST_METRICS_PORT

9090

API host mapping in bridge mode; actual API listener in host-NIC mode

HOST_TELEGRAF_PORT

9273

Prometheus host mapping in bridge mode; actual listener in host-NIC mode

HOST_TELEGRAF_HTTP_PORT

8186

HTTP ingest host mapping in bridge mode; actual listener in host-NIC mode

The default metrics-manager service uses bridge networking. metrics-manager-host-nic is opt-in via host-nic-telemetry and shares the host network. Stop the other variant and name the desired service explicitly: activating the profile alone selects both. See switching variants and exposure. The existing test profile is only a development test runner.

Compose supplies matching listener ports and local scrape/ingestion URLs for each variant; internal API/Prometheus/HTTP ports must be distinct integers from 1 through 65535. The entrypoint derives local URLs from port values for plain Docker unless explicitly overridden. If running Telegraf directly with the Docker configuration, export TELEGRAF_PORT and TELEGRAF_HTTP_PORT first. Custom configurations must match these listeners themselves; they are never rewritten.

Optional Docker Collectors#

These settings are read by the Docker entrypoint, not Pydantic. Compose passes them into the container; use -e with plain docker run. Optional flags accept only lowercase true/false (including rejecting empty values and auto). They control bundled additions, not the baseline or user-supplied inputs.

Variable

Default

Description

ENABLE_RAPL

false

Native package/platform RAPL power

ENABLE_POWERSTAT_EXTRA

false

Per-CPU frequency/temperature, without duplicating base package/C-state inputs

ENABLE_TURBOSTAT

false

Turbostat diagnostics with a bounded executable/MSR preflight

ENABLE_TEMPERATURE

false

High-resolution package temperature and min/max/mean

ENABLE_INTERRUPTS

false

Numbered device IRQs with per-CPU series

ENABLE_NETWORK

false (bridge), true (host-NIC)

Network traffic and NIC driver counters; explicit false overrides either default

ENABLE_NETWORK_VIRTUAL

false

Separate veth/lxc traffic and driver counters

ENABLE_STORAGE

false

Filesystem usage and disk I/O; bundled serial/WWID export disabled

TURBOSTAT_INTERVAL

5

Positive integer seconds fitting a Telegraf duration

TURBOSTAT_BIN

/usr/local/bin/turbostat

Absolute container executable path, without spaces/shell expansion

HOST_TURBOSTAT_BIN

(unset)

Host binary source for the explicit read-only mount in the optional Compose override

Invalid flags/interval/path syntax fail startup. Unavailable or incompatible Turbostat binaries and unreadable MSRs produce a warning and skip the automatic Turbostat input, not the whole stack. Docker bind failures occur before that check and can still prevent container creation. See the collectors guide for the mount, conflicts with custom inputs and hardware/namespace scope. Recreate containers after changing settings; there is no hot reload. Bare-metal auto semantics are unchanged.


Example Configurations#

Development Setup#

# .env
ENVIRONMENT=development
LOG_LEVEL=DEBUG
LOG_FORMAT=text
CORS_ORIGINS=*
RATE_LIMIT_ENABLED=false

High-Throughput Scenario#

# .env
RATE_LIMIT_REQUESTS_PER_MINUTE=5000
RATE_LIMIT_BURST=500
MAX_METRICS_IN_MEMORY=500000
PROMETHEUS_POLLER_INTERVAL_MS=200
ENABLE_GZIP_COMPRESSION=true

Production with Reverse Proxy#

# .env
ENVIRONMENT=production
LOG_LEVEL=WARNING
LOG_FORMAT=json
CORS_ORIGINS=https://my-dashboard.example.com,https://grafana.example.com
TRUST_FORWARDED_HEADERS=true
METRICS_MANAGER_HOSTNAME=production-node-01

Behind Corporate Proxy#

# .env
http_proxy=http://proxy.example.com:8080
HTTP_PROXY=http://proxy.example.com:8080
https_proxy=http://proxy.example.com:8080
HTTPS_PROXY=http://proxy.example.com:8080
no_proxy=localhost,127.0.0.1
NO_PROXY=localhost,127.0.0.1

Custom Metrics Scripts#

The image ships with a Telegraf inputs.exec block that, every 10 seconds, runs every executable *.sh and *.py file it finds in /app/custom-metrics/ and feeds the stdout straight into the Prometheus endpoint on :9273. For details, see Custom Metrics Scripts.

Simply dropping a script into the directory is the easiest way to publish a metric the service does not collect by default.

How the Directory is Wired#

  • The directory is created by the Dockerfile (/app/custom-metrics)

  • In compose.yaml it is mounted as a named volume custom-metrics:, so scripts survive container restarts

  • telegraf.conf ships this block (do not edit unless you know what you are doing):

[[inputs.exec]]
  commands = ["/bin/sh -c 'for f in /app/custom-metrics/*.sh /app/custom-metrics/*.py; do [ -f \"$f\" ] && [ -x \"$f\" ] && \"$f\"; done 2>/dev/null; true'"]
  timeout = "5s"
  data_format = "influx"
  interval = "10s"

Script Requirements#

Each script must:

  • Be executable (chmod +x)

  • Print InfluxDB Line Protocol on stdout, one metric per line

  • Finish within 5 seconds (Telegraf kills longer runs)

  • Produce clean output (no debug prints, banners, or stderr)

  • Handle errors gracefully (non-zero exit codes do not crash Telegraf)

Example: Fan RPM Metric#

See Custom Metrics Scripts for a complete end-to-end example.


Optional Components#

The Metrics Manager image includes optional components that are bundled but not active by default:

qmmd (Prometheus GPU Exporter)#

What it is: A lightweight Prometheus exporter for Intel Arc GPUs. It reads GPU metrics from sysfs and exposes them in Prometheus format.

Current Status: Bundled in the image but NOT started by default.

Why not enabled: The default Metrics Manager already collects GPU metrics via mm-plugin-gpu and Telegraf’s inputs.execd. Using both would be redundant.

When to enable: If you want a dedicated GPU metrics exporter that outputs to a separate Prometheus port (typically :9100 or similar) without going through Telegraf.

How to enable: Edit supervisord.conf or extend it in your downstream image:

[program:qmmd]
command=/usr/local/bin/qmmd
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
priority=40

License: Prometheus exporter for Intel® GPUs, published on crates.io under the MIT license. See https://crates.io/crates/qmmd.


Optional Services & Extending supervisord#

The container’s bundled supervisord.conf contains an [include] section:

[include]
files=/etc/supervisor/conf.d/*.conf

Any *.conf file dropped into /etc/supervisor/conf.d/ is picked up automatically at supervisord start, so you do not need to fork or edit supervisord.conf to add your own programs.

Pattern: Add a Program in Your Downstream Image#

FROM intel/metrics-manager:2026.3.0

# Drop additional supervisord program units into the include directory.
COPY my-extra-service.conf /etc/supervisor/conf.d/my-extra-service.conf

my-extra-service.conf:

[program:my-extra-service]
command=/usr/local/bin/my-extra-service --flag
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
priority=40   ; >30 = starts after metrics-manager (priority 20)

Verify the Extra Unit is Running#

docker exec metrics-manager supervisorctl -c /etc/supervisor/supervisord.conf status
# my-extra-service                 RUNNING   pid 47, uptime 0:01:23

Custom Telegraf Configuration#

Mount Custom Config#

# compose.yaml
services:
  metrics-manager:
    volumes:
      - ./my-telegraf.conf:/etc/telegraf/telegraf.conf:ro

Or via environment:

TELEGRAF_CONFIG=./my-telegraf.conf docker compose up

Additional Config Directory#

The native-enabled container loads the main configuration plus a private snapshot of mounted telegraf.d/**/*.conf files and enabled bundled additions. It passes that snapshot using --config-directory; mounted source files are not modified. Review old drop-ins before upgrading: earlier native-enabled images mounted this directory but did not load it. Keep only one agent, output and HTTP listener configuration, and avoid duplicate inputs.

See Additional collectors in the native-enabled Docker image for optional RAPL/Turbostat setup, customization and metric verification. Use the Docker flags listed above; Python-reader flag names are not aliases. native/metrics-options.conf applies to installed bare-metal services only.

Example: Disable GPU or NPU Metrics#

The default telegraf.conf registers both GPU (qmassa) and NPU readers as [[inputs.execd]] blocks. To disable:

# my-telegraf.conf (omit GPU/NPU inputs)
[agent]
  interval = "1s"

[[outputs.prometheus_client]]
  listen = ":9273"

[[inputs.cpu]]
[[inputs.mem]]
# GPU and NPU inputs omitted

Structured Logging#

Logs are output in JSON format by default for easy parsing by log aggregators:

{
  "timestamp": "2026-03-04T10:15:30.123456Z",
  "level": "INFO",
  "logger": "app.routes",
  "message": "Accepted metrics via batch",
  "correlation_id": "abc-123-def",
  "extra": {"count": 10}
}

Switch to human-readable format for development:

LOG_FORMAT=text docker compose up

Correlation IDs#

Every request is assigned a correlation ID for distributed tracing. Pass your own via header or receive an auto-generated UUID:

curl -H "X-Correlation-ID: my-trace-123" http://localhost:9090/api/v1/metrics
# Response header: X-Correlation-ID: my-trace-123

Correlation IDs appear in all log entries for request tracing.

Supporting Resources#

License#

Copyright (C) 2025-2026 Intel Corporation

SPDX-License-Identifier: Apache-2.0