Metadata

Overview

DL Streamer is transitioning to the GStreamer Analytics metadata library as the primary way to represent inference results. The legacy metadata API remains functional but is deprecated.

GStreamer Analytics Metadata

This is the recommended metadata API for new code. All metadata for a buffer is stored inside a single GstAnalyticsRelationMeta container with typed entries (GstAnalyticsMtd) connected by relations (CONTAIN, IS_PART_OF, RELATE_TO).

Key types:

Type	Description
GstAnalyticsODMtd	Object detection (bbox, label, confidence)
GstAnalytics3DODMtd	3D object detection (oriented box, class, confidence, sensor modality) — DL Streamer extension
GstAnalyticsClsMtd	Classification
GstAnalyticsTrackingMtd	Object tracking
GstAnalyticsKeypointMtd	Single keypoint
GstAnalyticsGroupMtd	Ordered group of metadata
GstAnalyticsSegmentationMtd	Semantic segmentation (class-index mask)
GstAnalyticsTensorMtd	Raw tensor payload — generic gvainference output tensors and instance-segmentation soft masks
GstAnalyticsKeypointDescriptor	Static keypoint layout registry — DL Streamer extension
GstAnalyticsZoneMtd	Zone presence (zone ID) — DL Streamer extension
GstAnalyticsTripwireMtd	Tripwire crossing (tripwire ID, direction) — DL Streamer extension

For the full API documentation, keypoint descriptor details, and code examples, see GStreamer Analytics Metadata.
You can also check out our post at discourse.gstreamer.org for input from GStreamer community.

Custom Watermark Metadata

gvawatermark automatically renders custom drawing primitives attached directly to GStreamer buffers using the DL Streamer watermark metadata types.

Key types:

Type	Description
WatermarkDrawMeta	Polygon or polyline defined by an ordered list of (x, y) coordinate pairs (max 128 pairs)
WatermarkCircleMeta	Circle defined by center (cx, cy), radius, color, and thickness
WatermarkTextMeta	Text label at position (x, y) with font, scale, color, and optional background

For the full API documentation, see Watermark Metadata.

3D Sensor Metadata

DL Streamer’s 3D pipelines (LiDAR and mmWave radar) carry metadata at two levels:

• 3D detections — oriented 3D bounding boxes are stored as GstAnalytics3DODMtd inside the buffer’s GstAnalyticsRelationMeta.

• Raw-sensor metadata — standalone GstMeta types attached directly to the buffer that describe the sensor frame and its low-level results:

Key types:

Type	Description
GstAnalytics3DODMtd	3D object detection — oriented box (x, y, z, l, w, h, yaw, pitch, roll), class, confidence, sensor modality.
LidarMeta	Raw LiDAR frame framing: point count, frame/stream id, timestamps. The point cloud itself lives in the buffer payload.
GstRadarProcessMeta	Radar signal-processing results: point clouds, clusters, and per-object tracks.

GstAnalytics3DODMtd is documented in GStreamer Analytics Metadata. For the raw-sensor LidarMeta and GstRadarProcessMeta types, see 3D Sensor Metadata.

Legacy Metadata (deprecated)

DEPRECATED: The legacy metadata API based on GstVideoRegionOfInterestMeta, GstGVATensorMeta, and GstGVAJSONMeta is deprecated and will be removed in the future. New code should use the GStreamer Analytics API described below.

The legacy API attaches GstVideoRegionOfInterestMeta to buffers with bounding box coordinates (x, y, w, h) and a GList *params of GstStructure entries holding additional inference results (detection confidence, classification labels, keypoints, etc.). Two additional custom metadata types are defined:

• GstGVATensorMeta — raw tensor output from gvainference

• GstGVAJSONMeta — JSON conversion output from gvametaconvert

For reference documentation of the legacy API, see Legacy Metadata.

Element input/output summary

Element	Description	Input	Output (Analytics)	Output (Legacy)
gvadetect (full-frame)	Object detection on full frame	GstBuffer	ODMtd, ClsMtd, GstAnalyticsGroupMtd, GstAnalyticsKeypointMtd, GstAnalyticsTensorMtd	ROI + GstStructure params
gvadetect (roi-list)	Object detection per ROI	GstBuffer + ROI + ODMtd	ODMtd, ClsMtd, GstAnalyticsGroupMtd, GstAnalyticsKeypointMtd, GstAnalyticsTensorMtd	ROI (with parent_id) + GstStructure params
gvaclassify (roi-list)	Object classification per ROI	GstBuffer + ROI + ODMtd	ClsMtd, GstAnalyticsGroupMtd, GstAnalyticsKeypointMtd, GstAnalyticsSegmentationMtd, GstAnalyticsTensorMtd	extended ROI params
gvaclassify (full-frame)	Full-frame classification	GstBuffer	ClsMtd, GstAnalyticsGroupMtd, GstAnalyticsKeypointMtd, GstAnalyticsSegmentationMtd, GstAnalyticsTensorMtd	GstGVATensorMeta
gvainference (full-frame)	Generic full-frame inference	GstBuffer	GstAnalyticsTensorMtd	GstGVATensorMeta
gvainference (roi-list)	Generic inference per ROI	GstBuffer + ROI + ODMtd	GstAnalyticsTensorMtd	extended ROI params
gvatrack	Object tracking	GstBuffer + ROI + ODMtd	TrackingMtd	ROI + object_id param
gvaanalytics	Zone and tripwire analytics	GstBuffer + ODMtd + TrackingMtd	ZoneMtd, TripwireMtd, WatermarkDrawMeta, WatermarkCircleMeta	—
gvametaconvert	Metadata → JSON	GstBuffer + ROI + ODMtd + ClsMtd + GstAnalyticsGroupMtd + GstAnalyticsKeypointMtd + TrackingMtd + GstAnalyticsSegmentationMtd + GstAnalyticsTensorMtd + ZoneMtd + TripwireMtd + 3DODMtd + LidarMeta + GstRadarProcessMeta + GstAnalyticsBatchMeta + GstGVATensorMeta	—	GstGVAJSONMeta
gvametapublish	JSON → MQTT/Kafka/File	GstBuffer + GstGVAJSONMeta	—	—
gvametaaggregate	Merge from multiple streams	GstBuffer + ROI + GstGVATensorMeta + ODMtd + ClsMtd + GstAnalyticsGroupMtd + GstAnalyticsKeypointMtd + GstAnalyticsSegmentationMtd + GstAnalyticsTensorMtd	ODMtd, ClsMtd, GstAnalyticsGroupMtd, GstAnalyticsKeypointMtd, GstAnalyticsSegmentationMtd, GstAnalyticsTensorMtd	ROI + GstStructure params, GstGVATensorMeta
gvawatermark	Overlay on video	GstBuffer + ROI + ODMtd + ClsMtd + GstAnalyticsGroupMtd + GstAnalyticsKeypointMtd + TrackingMtd + GstAnalyticsSegmentationMtd + GstAnalyticsTensorMtd + GstGVATensorMeta + WatermarkDrawMeta + WatermarkCircleMeta + WatermarkTextMeta	—	—
gvagenai	VLM inference on video frames	GstBuffer	ClsMtd	GstGVAJSONMeta
gvaaudiotranscribe	Speech recognition (Whisper)	GstBuffer (audio)	ClsMtd	—
gvastreammux	Merge streams into one batch	GstBuffer (per stream) + metadata	GstAnalyticsBatchMeta (container)	—
gvastreamdemux	Split batch back to per-stream buffers	GstAnalyticsBatchMeta (container)	per-stream GstBuffer + metadata	—
g3dradarprocess	mmWave radar signal processing	GstBuffer	GstRadarProcessMeta	—
g3dlidarsrc	Live LiDAR capture from a device → point cloud	— (source element)	LidarMeta	—
g3dlidarparse	Parse raw LiDAR frame → point cloud	GstBuffer	LidarMeta	—
g3dinference	3D detection on LiDAR	GstBuffer + LidarMeta	3DODMtd	—
g3dobjectfuser	Fuse camera 2D with 3D (LiDAR/radar)	GstAnalyticsBatchMeta + ODMtd + 3DODMtd + GstRadarProcessMeta	3DODMtd + TrackingMtd	—
g3drender	Render camera 2D + 3D point cloud, overlay detections	GstAnalyticsBatchMeta + ODMtd + 3DODMtd + TrackingMtd	—	—