Get Started#
Prerequisites#
./install.sh automatically runs ./startup.sh first, which
installs/verifies curl/git, Docker, Node.js/npm (via nvm), and Python
(venv + PyYAML) — there is only one script to run. startup.sh is idempotent
(safe to re-run; skips whatever is already present) and can also be run on
its own if you want finer-grained control (--skip-base, --skip-docker,
--skip-nodejs, --skip-python) or want to pre-provision a host before
installing anything:
./startup.sh --help
Notes on what it does with Docker:
If
HTTP_PROXY/HTTPS_PROXY/NO_PROXY(either case) are set, it also writes them into the Docker daemon’s own systemd config (/etc/systemd/system/docker.service.d/http-proxy.conf, root-only) so the daemon can pull images through the proxy, and restarts the daemon when that changes. This needs sudo; without it the step is skipped with a warning.Before installing anything, it rejects an
HTTP_PROXY,HTTPS_PROXYorALL_PROXYvalue (either case) that contains a quote, whitespace or a control character, for example a stray trailing". Such a value is forwarded into the containers, and agent HTTP clients refuse to start with it. The error names the variable, never its value.If it just added your user to the
dockergroup, it restarts itself under that group automatically, so no new terminal is needed. Only if that cannot be done (sgandsudoboth unavailable) does it exit with code2and ask you to open a new terminal (or runnewgrp docker) and re-run.
What it does#
Host prep — runs
startup.sh(curl/git, Docker, Node.js, Python), then an Intel GPU check (PCI vendor8086,i915/xedriver,/dev/drirender nodes, Level-Zero/OpenCL compute runtime readiness — assumed already provisioned, e.g., via an Edge Pack, not installed by this step). Host users and groups are not modified: containers get render-node access through--deviceand--group-add.Inference backend — requires a model: pass
--model(the installer prompts for it on a terminal and fails otherwise, unless a model is already exported from an earlier run). Starts OpenVINO Model Server (Docker, GPU passthrough) and fetches that model from its hub (see Model sources). By default a Hugging Face id such asorg/nameis converted to OpenVINO IR via OVMS’s own “-py” Docker image in--pullmode — unlike plainoptimum-cli, that also generates the MediaPipe graph OVMS’s/v3/chat/completionsendpoint needs. Pre-convertedOpenVINO/*models are only downloaded, by the same OVMS image that serves them. For gated Hugging Face models, exportHF_TOKENfirst. SetHARNESS_OVMS_EXPORTER=export-model-pyor=hf-snapshot(a plain download, no conversion, for HF repos already published as ready-to-serve OpenVINO IR) for alternate exporters (see Known limitations). Always also starts the Inference Router (IR) — every inference request goes through it. The router is the publishedintel/inference-routerimage, pulled once (setHARNESS_IR_BUILD_FROM_SOURCE=1to build it from a pinned commit ofmicroservices/inference-routerinopen-edge-platform/edge-ai-librariesinstead). By default IR’s one configured provider is the local OVMS. SetHARNESS_LLM_ROUTER_ENDPOINT(and, if it needs one,HARNESS_LLM_ROUTER_API_KEY) to point IR at an existing external OpenAI-compatible endpoint instead; OVMS is then not started, and--modelnames the model that endpoint serves.Agent install — installs the selected agent harness via its own official installer/package (
openclaw,dcode, or Hermes’s installer), wired to the Inference Router’s endpoint.Onboarding — wires the installed agent to that endpoint and checks it:
OpenClaw: registers the endpoint as a custom
ovmsmodel provider (openclaw config set models.providers.ovms …), sets it as the default model, and runs a smoke test. Ifopenclaw config setfails (e.g., its Gateway daemon is running), it prints the exact manual command instead.Deep Agents Code (
dcode): writes its provider config and runs a smoke test; the smoke test is reported as failed if dcode prints an error, even when it exits 0.Hermes: prints the endpoint and where to point a provider in its own config (
~/.hermes/config.yaml); there is no automated smoke test. If Hermes already points at this endpoint with a different model name than the one served, it warns and prints thehermes config set model.default <name>fix.
Published ports (OVMS and the Router) bind to loopback
(127.0.0.1) by default, so nothing is reachable from other machines unless
you opt in with HARNESS_ADVERTISED_HOST (or the prompt that sets it) or
HARNESS_BIND_HOST.
Usage#
./install.sh --harness openclaw --model <org/name>
For example, with a small Hugging Face model (any supported model works):
./install.sh --harness openclaw --model Qwen/Qwen2.5-0.5B-Instruct
The model is served under its name without the organisation prefix, so the
command above serves Qwen2.5-0.5B-Instruct. The Inference Router matches
model names exactly: a client that requests Qwen/Qwen2.5-0.5B-Instruct gets
Unknown model. The installer prints the served name after the install, and
curl http://<host>:8010/v1/models (default port) lists it.
./install.sh --harness deepagents-code --model <org/name> --non-interactive
Or configure via environment variables:
export HARNESS_AGENT=openclaw
export HARNESS_HF_MODEL=<org/name>
./install.sh --non-interactive
For example:
export HARNESS_AGENT=openclaw
export HARNESS_HF_MODEL=Qwen/Qwen2.5-0.5B-Instruct
./install.sh --non-interactive
Options: --non-interactive, --skip-gpu-check, --harness <name>,
--model <model-name>, --model-hub <hub>, --model-config <json>,
--model-revision <rev>, --model-type <type>,
--model-precision <int4|int8|fp16|fp32>, --model-device <device>; run
./install.sh --help for the full list of options and environment variables.
Model sources#
--model is the model’s name as its hub names it, and --model-hub says
which hub to fetch it from. Without --model-hub the hub is openvino: a
Hugging Face id that is converted to OpenVINO IR. Any other hub selects the
Model Download service’s ephemeral container (get_model.sh, pinned and
checksum-verified) on its own — there is no need to set
HARNESS_OVMS_EXPORTER.
|
|
Extra input |
Served to agents |
|---|---|---|---|
|
Hugging Face id, |
|
yes |
|
Hugging Face id, |
|
only if the repo already contains an OVMS |
|
plain name substituted for |
|
only if the tarball contains a |
|
|
no (OVMS cannot load Ollama models yet) |
|
|
model name or a comma-separated list |
|
no |
|
Geti project name |
|
no |
|
model name or a comma-separated list |
no |
|
|
any label |
|
no |
A model is served — registered with OVMS and wired to the Inference Router
— only when the download contains an OVMS graph.pbtxt. Everything else is
downloaded and tracked (./install.sh models list shows it as “not served by
OVMS”), and the installer says why agents will not use it. The router stays
on whatever OVMS already serves, so one model is served at a time.
# default: Hugging Face id, converted to OpenVINO IR
./install.sh --harness openclaw --model <org/name>
# Ollama: downloaded and tracked, not served by OVMS yet
./install.sh --harness openclaw --model-hub ollama --model <name:tag>
# a model tarball from an allowed URL
./install.sh --harness openclaw --model-hub remote-url --model <name> \
--model-config '{"url":"https://github.com/open-edge-platform/edge-ai-resources/raw/main/{name}.tar"}'
Options that an exporter does not use (for example --model-device with the
default exporter) are reported as ignored.
Other commands:
./install.sh status # host, OVMS/Router and agent health (read-only)
./install.sh onboard # re-run onboarding only
./install.sh models list # exported/downloaded models
./install.sh models remove <name> # delete one (and its OVMS config entry)
./install.sh mcp register <name> <url> [agent] # register an MCP endpoint (Hermes, OpenClaw, or HARNESS_MCP_REGISTER_CMD)
./install.sh skill <install|list|remove> [path|name] # Hermes skills only
See Run Without a Checkout, and Uninstall for the rest of the usage surface.