# Scene Controller Service
Scene Controller Microservice fuses multimodal sensor data to enable spatial analytics at the
edge for multiple use cases.
## Overview
The Scene Controller Microservice answers the fundamental question of `What, When and Where`. It receives object detections from multimodal inputs (primarily multiple cameras), contextualizes them in a common reference frame, fuses them and tracks objects over time.
The Scene Controller's output provides various insights for the tracked objects in a scene, including location, object visibility across cameras, velocity, rotation, center of mass. Additionally, base analytics like regions of interest, tripwires, and sensor regions are supported out of the box to enable developers to build their applications quickly and realize business goals.
To deploy the scene controller service, refer to the [Get Started](./get-started.md) guide. The service supports configuration through specific arguments and flags, which default to predefined values unless explicitly modified.
### Configurable Arguments and Flags
`--maxlag`: Maximum allowable delay for incoming messages. If a message arrives more than 1 second late, it will be discarded by the Scene Controller. This threshold can be adjusted to accommodate longer inference times, ensuring no messages are discarded. Discarded messages will appear as "FELL BEHINDS" in the service logs.
`--broker`: Hostname or IP of the MQTT broker, optionally with `:port`.
`--brokerauth`: Authentication credentials for the MQTT broker. This can be provided as `user:password` or as a path to a JSON file containing the authentication details.
`--resturl`: Specifies the URL of the REST server used to provide scene configuration details through the REST API.
`--restauth`: Authentication credentials for the REST server. This can be provided as `user:password` or as a path to a JSON file containing the authentication details.
`--rootcert`: Path to the CA (Certificate Authority) certificate used for verifying the authenticity of the server's certificate.
`--cert`: Path to the client certificate file used for secure communication.
`--ntp`: NTP server.
`--tracker_config_file`: Path to the JSON file containing the tracker configuration. This file is used to enable and manage time-based parameters for the tracker.
`--reid_config_file`: Path to the JSON file containing Re-ID (Re-Identification) configuration. This file controls Re-ID specific settings such as stale feature timeout, feature accumulation thresholds, similarity metric selection, and similarity scoring. See [Extended Re-ID](./Extended-ReID.md) for details.
When `similarity_metric` is `L2` (default), Re-ID vectors follow distance-style matching semantics (lower values are better). When `similarity_metric` is `COSINE`, Re-ID vectors are normalized before write/query and SceneScape uses VDMS `IP` internally; similarity scores are expected in `[-1, 1]`.
`--schema_file`: Specifies the path to the JSON file that contains the metadata schema. By default, it uses [metadata.schema.json](https://github.com/open-edge-platform/scenescape/blob/main/controller/src/schema/metadata.schema.json). This schema outlines the structure and format of the messages processed by the service.
`--visibility_topic`: Specifies the topic for publishing visibility information, which includes the visibility of objects in cameras. Options are `unregulated`, `regulated`, or `none`.
`--analytics-only`: Enables analytics-only mode (experimental feature). In this mode, the Scene Controller consumes tracked objects from a separate Tracker service via MQTT instead of performing tracking internally. The tracker is not initialized, and camera/scene data processing is skipped. Child scenes are not supported. Note: `--pose-adjustment` has no effect in this mode, since pose adjustment is applied during camera data processing which is skipped in analytics-only mode. This mode can also be enabled via the `CONTROLLER_ENABLE_ANALYTICS_ONLY` environment variable set to `true`.
`--pose-adjustment`: Enables pose-based bounding box adjustment before world projection. When enabled, the controller uses pose keypoints (e.g. from a `yolo11n-pose` model) to refine the bounding box used for projecting detections into world coordinates. This is disabled by default. Not supported in `--analytics-only` mode. Cannot be used together with Extended ReID (VDMS-based cross-camera re-identification); see [Extended Re-ID](./Extended-ReID.md) for details. Can also be enabled via the `CONTROLLER_ENABLE_POSE_ADJUSTMENT` environment variable set to `true`. Requires the DL Streamer video pipeline to use a pose estimation model that provides keypoint data. See the [DL Streamer Pipeline Server documentation](../../../dlstreamer-pipeline-server/README.md#enable-pose-estimation) for pipeline setup.
`--pose_adjustment_config_file`: JSON file that defines pose-adjustment label routing. The default file is `pose-adjustment-route.json` next to the controller executable. Use this file to map each registered pose-adjustment strategy label to the incoming labels that should dispatch to it.
Example `pose-adjustment-route.json`:
```json
{
"person": ["human", "pedestrian"],
"vehicle": ["car", "truck", "sedan"]
}
```
Resolution order is: exact label, then configured route labels. Routes are flattened at startup so message-time dispatch remains a direct lookup.
### Configuration
For detailed configuration guidance:
- Tracker configuration: See [How to Configure the Tracker](./how-to-configure-tracker.md)
- Re-ID configuration: See [Extended Re-ID](./Extended-ReID.md)
## Input/Output Message Formats
For details on the MQTT message formats accepted and produced by the Scene Controller, see [SceneScape Controller Data Formats](./data_formats.md).
## Architecture
Figure 1: Architecture Diagram
## Sequence Diagram: Scene Controller Workflow
The Client receives regulated scene detections via MQTT, which are the result of processing and filtering raw detections. The pipeline begins when the Scene Controller Microservice receives detections from the camera. It processes these to track moving objects, then publishes scene detections and events through MQTT. These messages may include both regulated (filtered and formatted) and unregulated (raw) scene detections. A Multi Object Tracker Loop is involved in managing detections within MQTT.
_Figure 2: Scene Controller Sequence diagram_
## Supporting Resources
- [Get Started Guide](./get-started.md)
- [How to Configure the Tracker](./how-to-configure-tracker.md)
- [Extended Re-ID](./Extended-ReID.md)
- [Data Formats](./data_formats.md)
- [API Reference](./api-reference.md)
:::{toctree}
:hidden:
get-started.md
how-to-configure-tracker.md
Extended-ReID.md
API Reference
data_formats.md
pose_adjustment.md
:::