Positioning Service#
The Positioning service is the deployment boundary for live source-pose capabilities.
The service shell connects to MQTT, publishes resolver status, and exposes health
checks. When PTZ sources are configured, it polls each ONVIF PTZ camera, passes raw
PTZSample messages in-process to the PTZ Resolver, and publishes one resolved
PoseContext per accepted sample.
Note: The Scene Controller can consume resolved poses when you enable its PTZ mode; see PTZ Pose Consumption. The Tracker does not consume poses yet. With the Controller mode off, publishing poses does not change how detections are tracked.
How It Works#
The service connects to the standard MQTT broker TLS listener on port 1883 and
verifies the broker against the SceneScape CA certificate. It uses
positioning.auth like other SceneScape services. After it
connects, it publishes a retained status message and registers an offline last-will
message on:
scenescape/positioning/status/{resolver_id}
The payload contains resolver_id, status (online or offline), and
timestamp. For the last-will message, the timestamp is the session start time,
not the time the connection was lost.
The service writes only its resolver status topic and the resolved-pose topics of
its assigned sources (scenescape/positioning/pose/{source_id}), and subscribes
only to camera configuration invalidations for those assigned sources
(scenescape/cmd/camera/config/{camera_id}). Source ownership is validated at
startup against the shared assignment manifest. The container reports healthy
only while it is connected to the broker.
PTZ polling#
One poller thread per configured source calls ONVIF PTZ GetStatus every
poll_interval_s (default 0.2 s, 5 Hz), including while the camera is stationary.
If a call overruns its tick, missed ticks are skipped; no burst, retry queue, or
replay follows. Each reading becomes one generated scenescape.ptz.v1.PTZSample
and is passed in-process to the sample sink:
Field |
Value |
|---|---|
|
Configured |
|
New UUID each time the service starts a poller; unchanged across device reconnects |
|
Starts at 1 and increments by one per emitted sample within |
|
Host wall-clock midpoint of the GetStatus request/response interval |
|
Half the measured round trip, rounded up (never 0) |
|
Raw pan/tilt (and zoom when reported); the reported space URI and its advertised |
|
Advisory only; from |
If the device reports a position space that its PTZ node does not advertise, the
ranges are NaN so the Resolver can reject the sample. The poller applies no
calibration math and never writes to Manager or MQTT. After max_consecutive_failures
failed polls (default 5), it reconnects with exponential backoff capped at 30 s.
Move events#
Some cameras publish a vendor “is moving” property event. For example, Axis cameras
publish tns1:PTZController/tnsaxis:Move/Channel_1 with the is_moving data item.
When move_event is configured, a separate thread holds an ONVIF PullPoint
subscription, renews it, and always unsubscribes on shutdown, because devices limit
concurrent PullPoints. Each state change is logged as a structured
ptz_move_event JSON record containing source_id, is_moving,
property_operation, device_utc_time, and received_at. These records are
evidence for calibration. If move_status is event, the latest event state also
sets reported_move_status. Move events never establish pose validity: the Resolver
derives motion from position history.
ONVIF client#
The poller uses a small in-tree SOAP 1.2 client
(positioning/src/positioning/onvif/) built on requests and defusedxml, both
pinned with hashes in requirements-runtime.txt. It implements only
GetSystemDateAndTime, GetCapabilities, GetDeviceInformation, GetProfiles,
GetNodes, GetStatus, GetEventProperties, and PullPoint create, pull, renew, and
unsubscribe operations. These alternatives were evaluated:
Option |
Decision and reason |
|---|---|
|
Rejected: last released in 2018 and unmaintained. It needs monkey patches for drifting device clocks and Axis subscription reference parameters. |
|
Not selected: it is maintained, but it is async-only and pulls in zeep, lxml, httpx, and their transitive tree, which must all be hash-pinned. It also loads WSDLs at startup, while the poller needs fewer than ten operations. |
|
Rejected: it is not published as a package, and the plan forbids cloning or vendoring source during an image build. |
In-tree SOAP ( |
Selected: two small, actively maintained dependencies that support Python 3.11. Device XML is parsed with DTDs and entities forbidden, and responses are limited to 1 MiB. The client times the actual HTTP exchange for the midpoint timestamp. |
Authentication uses WS-Security UsernameToken PasswordDigest. The device clock
offset is read with an unauthenticated GetSystemDateAndTime and resynchronized after
a NotAuthorized fault, so a drifting camera clock does not block authentication.
HTTP Digest is used only if the device sends a 401 challenge. Host HTTP proxy
variables are ignored for camera traffic. Service addresses advertised by the device
keep their path and port, but the configured camera host is always used, so
credentials are never sent to a different host. Plain http sends a digest rather
than the password; use https where the camera supports it, and keep cameras on an
isolated network.
The design rationale is in the PTZ pose design.
Pose resolution#
The Resolver validates every sample before accepting it: finite position and ranges,
position inside the advertised ranges, a position space URI and ranges matching the
calibration, timestamp uncertainty within max_timestamp_uncertainty_s, and a
strictly increasing sequence. Duplicate, out-of-order, malformed, or unassigned
samples produce no PoseContext; consumers see them as sequence gaps. A new
adapter_instance_id (poller restart) or a sequence gap resets motion and backlash
history. A newer sequence whose timestamp does not increase indicates a backward
host-clock step: the Resolver accepts it as invalid_sample and resets timestamp
continuity together with motion and backlash history.
Every accepted sample publishes exactly one retained QoS 0 PoseContext on
scenescape/positioning/pose/{source_id}. A pose is valid only when all of these hold:
The clock is synchronized (see Clock synchronization).
The sample is younger than
max_pose_age_s; otherwisesample_stale.A Manager calibration is loaded; otherwise
calibration_missing.Backlash is known on both axes. After a restart, gap, or calibration change, an axis becomes known only when the camera reports a position within
backlash_home_tolerance_degof its calibrated home; otherwisebacklash_unknown.The camera is stationary: no axis moved more than
settle_angular_threshold_degbetween samples (slewing, reasonmoving), and the history covers a fullsettle_interval_swindow without motion (settling).The calibration’s
reprojection_rms_pxdoes not exceedtarget_reprojection_rms_px, and the Manager home pose is a rigid transform.
The resolved orientation composes backlash-corrected pan and tilt deltas about the
calibrated axes in the calibration’s composition_order. Each axis’s home reference
uses the calibrated approach_direction (ONVIF position travel direction) and the
curve’s angular direction: the physical angle trails the reported home angle by half the
home backlash. Continuing in that direction moves immediately; a reversal consumes the
full slack. Translation, intrinsics, and distortion stay those of the home calibration.
Calibration reload#
Calibration comes only from Manager. When Manager publishes
scenescape/cmd/camera/config/{camera_id} for an assigned source, the Resolver marks
that source reload-pending and publishes invalid poses with calibration_changed.
It retains the cached calibration and motion continuity while the read is in flight.
An identical calibration reload restores valid pose publication without requiring a
return to home; a changed calibration resets motion and backlash history, so a valid
pose again requires known backlash and a full settle window. Reloads run on a
background worker, not in the MQTT callback. Every calibration_refresh_interval_s
the service also rereads all sources, which retries failed reloads and recovers from a
missed notification. This refresh follows its own monotonic schedule, so frequent
notifications for one source cannot postpone it.
Each invalidation advances a per-source generation. A reload captures the generation before reading from Manager and applies its result only if the generation is still current, so a read that was in flight during an invalidation cannot restore the old calibration; the queued reload fetches the replacement.
Positioning configures a 10-second connection and read-inactivity timeout on its
Manager RESTClient; authentication and camera reads both use that timeout. A
timeout follows the existing calibration failure handling and permits a later
refresh to retry; it is not a total deadline for the complete calibration load.
Timeouts and connection failures during camera reads preserve the current calibration
state (a source already pending reload stays invalid). Definitive rejections, including
a deleted camera, identity mismatch, invalid calibration, or insufficient runtime
history capacity, clear the calibration and invalidate the source even during a
periodic refresh without a configuration notification. Rejections use the same
generation guard as successful reloads, so stale failures cannot clear a newer calibration.
Clock synchronization#
Poses are timestamped with the host clock, so the service checks it against the NTP
server given by --ntp (default: the POSITIONING_NTP_SERVER environment variable)
at startup and every 30 s, and logs the measured offset. While the absolute offset
exceeds max_clock_offset_s, no NTP server is configured, or no successful
measurement is younger than 90 s, every source publishes invalid poses with reason
invalid_sample; motion history and timestamp continuity are discarded, while
sequence ordering still rejects duplicates and replays. A failed or non-finite NTP
request also invalidates synchronization immediately, regardless of the age of the
previous measurement. The service keeps running and recovers automatically once a
successful in-range measurement returns, including after a backward clock correction.
Runtime metrics#
Every 60 s the service emits one structured positioning_metrics JSON record at
DEBUG level. Set the Positioning log level to DEBUG to make these records visible;
at the default INFO level they are intentionally quiet. Each histogram snapshot
contains cumulative bucket counts, observation count, and sum:
Metric |
Type |
Labels |
|---|---|---|
|
Histogram |
|
|
Counter |
|
|
Counter |
|
|
Histogram (time spent) |
|
|
Info |
|
|
Gauge |
- |
Dwell-rejection and projection-drift metrics belong to the Controller and Tracker consumers, which see detections; the Positioning service does not.
Build#
Build the runtime image from the repository root:
make positioning
The image is also built by make build-core and make build-all. Generate the
broker account with:
make init-secrets
This creates manager/secrets/positioning.auth, which is also used as the Manager
REST API credential.
Configure#
The service reads /app/config/positioning.json:
{
"resolver_id": "positioning-default"
}
Field |
Description |
|---|---|
|
Identifies this service instance in its status topic. Must be a single MQTT topic segment (no |
PTZ sources are optional and come from deployment configuration. The checked-in configuration has none:
{
"resolver_id": "positioning-default",
"poll_interval_s": 0.2,
"max_pose_age_s": 0.5,
"max_timestamp_uncertainty_s": 0.01,
"max_clock_offset_s": 0.05,
"target_reprojection_rms_px": 3.0,
"sample_ring_buffer_size": 64,
"calibration_refresh_interval_s": 30.0,
"sources": {
"<manager-camera-uid>": {
"type": "onvif_ptz",
"host": "<camera-host>",
"scheme": "https",
"port": 443,
"credentials_file": "/run/secrets/onvif/<manager-camera-uid>.json",
"move_status": "event",
"move_event": { "topic": "PTZController/Move", "data_item": "is_moving" }
}
}
}
Field |
Default |
Description |
|---|---|---|
|
|
Positive poll period for every source |
|
- |
Key is the |
|
required |
|
|
-, |
Camera ONVIF endpoint; |
|
|
HTTPS is the default; optionally provide the CA bundle path for a private camera certificate |
|
first PTZ |
Media profile whose PTZ configuration is polled |
|
required |
Mounted secret: |
|
|
Per-request HTTP timeout |
|
|
Source of advisory |
|
- |
|
|
required with sources |
Positive maximum sample age; at least |
|
required with sources |
Finite nonnegative upper bound for poller timestamp uncertainty |
|
required with sources |
Positive bound on the measured NTP offset; larger offsets invalidate all poses |
|
required with sources |
Positive calibration-quality limit; larger Manager residuals invalidate poses |
|
|
Must cover the largest settle/backlash history: |
|
|
Positive retry interval for Manager calibration loads |
Unknown keys are rejected at startup. Device credentials must never be committed. Keep them in deployment secrets mounted read-only into the container.
Source ownership is recorded separately in deployment/positioning-assignments.json
as a source_id to resolver_id mapping. The image contains the empty default at
/app/deployment/positioning-assignments.json; mount the deployment manifest over it.
At startup, every configured source must be assigned to this resolver_id, and every
entry assigned to this resolver_id must be configured; otherwise the service exits.
The Scene Controller reads the same manifest; see
Source Ownership.
Manager remains authoritative for camera-to-scene membership and camera calibration.
The service loads each camera’s ptz_calibration, computed pose_matrix, and scene
through the read-only Manager API. It subscribes only to configuration invalidations
for its assigned sources and retries calibration reads periodically. Failed or
incomplete calibration keeps the source invalid; a live pose is never used as a home
fallback. Accepted samples publish retained QoS 0 contexts on
scenescape/positioning/pose/{source_id}; malformed raw samples produce no context.
The service accepts these command-line options:
Option |
Default |
|---|---|
|
|
|
|
|
none (optional client certificate) |
|
none (optional client private key) |
|
|
|
|
|
defaults to |
|
|
|
|
|
|
Deploy#
On Kubernetes, enable the service with positioning.enabled=true and configure
sources, resolver limits, assignments, ONVIF credential Secrets, and camera
network allowlists through the
positioning.* Helm values. See
Deploy Scenescape on Kubernetes
for the related Helm values. The chart runs one replica, because only one instance may
publish poses for a given camera.
To roll back, disable the Positioning deployment or remove its sources. Without
sources the service publishes no poses. With the Controller PTZ mode off (the default),
either change leaves tracking unchanged; Tracker does not consume poses yet. To roll
back the Controller consumer, turn off its PTZ mode instead; see
PTZ Pose Consumption.