g3drender#
Renders a LiDAR point cloud — with optional camera streams and cross-modal detection metadata — into a BGR video frame.
Overview#
The g3drender element is a GstBaseTransform that accepts either a raw LiDAR stream or a GstAnalyticsBatchMeta container (produced by gvastreammux, followed by g3dobjectfuser or g3dbevfusion) and produces a single video/x-raw(BGR) frame per input buffer. It supports three rendering modes selected by the view-mode property.
Rendering modes:
bev(default) — Bird’s-eye view. The point cloud is projected top-down onto the XY plane. A metric grid and a coordinate-axis indicator are overlaid. 3D detection boxes are drawn as projected footprint polygons.perspective— Synthetic 3D camera view. Camera position is controlled bycam-distance,cam-elevation, andcam-azimuth. Each LiDAR point is coloured by height. 3D detection boxes are drawn as full wire-frame cuboids.cam-proj— LiDAR projected onto a real camera image. LiDAR points and 3D boxes are projected using KITTI-style calibration matrices (P2 × R0 × Tr) received fromg3dobjectfuserorg3dbevfusionvia theg3d/calibrationsticky event.
In bev and perspective modes, each camera tile also shows the 3D boxes projected into that camera, when the g3d/calibration event carries a calibration for the camera’s gvastreammux pad. Box edges are clipped at 0.5 m in front of the camera, so a box that is partly behind it keeps its visible part.
Properties#
Property |
Type |
Range |
Default |
Description |
|---|---|---|---|---|
|
int |
[1, 32767] |
800 |
Output frame width in pixels. Automatically doubled to |
|
int |
[1, 32767] |
800 |
Output frame height in pixels. |
|
enum |
0 / 1 / 2 |
0 |
Rendering mode: |
|
int |
[1, 20] |
2 |
Rendered radius of each LiDAR point in pixels. |
|
int |
[1, 100] |
16 |
Point cloud decimation factor. One point in N is rendered; higher values reduce density and improve throughput. The chosen point shifts by one each time, so LiDARs that interleave their laser rings in the point order (such as nuScenes) keep all rings. |
|
float |
[0.1, 20.0] |
1.0 |
BEV viewport zoom. |
|
float |
[1.0, 500.0] |
35.0 |
Synthetic camera distance from the scene origin in meters. Only effective in |
|
float |
[5.0, 89.0] |
30.0 |
Synthetic camera elevation angle in degrees ( |
|
float |
[-360.0, 360.0] |
180.0 |
Synthetic camera horizontal azimuth in degrees. Only effective in |
|
float |
[10.0, 150.0] |
60.0 |
Synthetic camera vertical field of view in degrees. Only effective in |
|
int |
[0, 255] |
0 |
|
|
bool |
true / false |
true |
Convert the camera background to grayscale in |
|
float |
[0.0, 1.0] |
0.65 |
Brightness multiplier for the camera background in |
|
enum |
|
|
LiDAR axis that points in the vehicle’s forward direction: |
Pipeline Examples#
LiDAR-only BEV render#
# BEV with 2× zoom; render every 16th LiDAR point with a radius of 2 pixels
gst-launch-1.0 -v \
multifilesrc location="<lidar_path>/lidar.bin" ! \
g3dlidarparse ! \
g3dinference ! \
g3drender view-mode=bev zoom=2.0 point-stride=16 point-radius=2 ! \
videoconvert ! \
autovideosink
Camera + LiDAR fusion with perspective render#
# Perspective mode: camera at 35 m distance, 30° elevation, 180° azimuth (directly behind), 1600×800 output
gst-launch-1.0 -v \
rtspsrc ! decodebin ! videoconvert ! gvadetect ! mux.sink_0 \
multifilesrc location="<lidar_path>/lidar.bin" ! g3dlidarparse ! g3dinference ! mux.sink_1 \
gvastreammux name=mux ! \
g3dobjectfuser ! \
g3drender view-mode=perspective width=1600 height=800 \
cam-azimuth=180 cam-elevation=30 cam-distance=35 ! \
videoconvert ! \
autovideosink
Camera + LiDAR fusion with cam-proj render#
cam-proj mode requires a camera stream and calibration matrices from g3dobjectfuser. The first camera stream is used by default; set cam-proj-index to select a different one.
# cam-proj mode: use the first camera stream as background; render every 4th LiDAR point with a radius of 2 pixels
gst-launch-1.0 -v \
rtspsrc ! decodebin ! videoconvert ! gvadetect ! mux.sink_0 \
multifilesrc location="<lidar_path>/lidar.bin" ! g3dlidarparse ! g3dinference ! mux.sink_1 \
gvastreammux name=mux ! \
g3dobjectfuser ! \
g3drender view-mode=cam-proj cam-proj-index=0 point-stride=4 point-radius=2 ! \
videoconvert ! \
autovideosink
Multi-camera + LiDAR fusion with perspective render#
# Perspective mode: camera at 35 m distance, 30° elevation, 180° azimuth (directly behind), 1600×800 output
gst-launch-1.0 -v \
rtspsrc ! decodebin ! videoconvert ! gvadetect ! mux.sink_0 \
rtspsrc ! decodebin ! videoconvert ! gvadetect ! mux.sink_1 \
multifilesrc location="<lidar_path>/lidar.bin" ! g3dlidarparse ! g3dinference ! mux.sink_2 \
gvastreammux name=mux ! \
g3dobjectfuser ! \
g3drender view-mode=perspective width=1600 height=800 \
cam-azimuth=180 cam-elevation=30 cam-distance=35 ! \
videoconvert ! \
autovideosink
Input / Output#
Capability |
|
|---|---|
Sink |
|
Sink |
|
Src |
|
The element always allocates a new output buffer; the input buffer is not modified.
Canvas layout:
# LiDAR only
┌──────────────────────────┐
│ LiDAR panel │
│ (height × height) │
└──────────────────────────┘
# LiDAR + 1 camera
┌──────────────────────────┬──────────────────────────┐
│ camera panel │ LiDAR panel │
│ (remaining width) │ (height × height) │
└──────────────────────────┴──────────────────────────┘
# LiDAR + 2 cameras (1 col × 2 rows)
┌──────────────────────────┬──────────────────────────┐
│ camera 0 │ LiDAR panel │
│──────────────────────────│ (height × height) │
│ camera 1 │ │
└──────────────────────────┴──────────────────────────┘
# LiDAR + 3 cameras (1 col × 3 rows)
┌──────────────────────────┬──────────────────────────┐
│ camera 0 │ LiDAR panel │
│──────────────────────────│ (height × height) │
│ camera 1 │ │
│──────────────────────────│ │
│ camera 2 │ │
└──────────────────────────┴──────────────────────────┘
# LiDAR + 4 cameras (2 cols × 2 rows)
┌─────────────┬────────────┬──────────────────────────┐
│ camera 0 │ camera 1 │ LiDAR panel │
│─────────────┼────────────│ (height × height) │
│ camera 2 │ camera 3 │ │
└─────────────┴────────────┴──────────────────────────┘
# cam-proj mode (full canvas, any number of cameras)
┌──────────────────────────────────────────────────────┐
│ camera background + LiDAR projection │
│ (width × height) │
└──────────────────────────────────────────────────────┘
The LiDAR panel is always a height × height square anchored to the right edge. Each camera frame is letterboxed into its grid cell. The layout switches from 1 column to 2 columns when 4 or more camera streams are present.
Notes#
BEV range and zoom: LiDAR has no native image resolution. In
bevmode the internal render range defaults to ±50 m on each axis, mapped to the output canvas.zoom=2.0halves the visible range (25 m), zooming in;zoom=0.5doubles it (100 m), zooming out.zoomhas no effect inperspectiveorcam-projmode.Auto-double width: When the input is a batch stream and
width == height,g3drenderautomatically doubleswidthtoheight × 2during caps negotiation so the default square canvas fits the side-by-side camera + LiDAR layout. Setwidthto a value other thanheightto suppress this behaviour.Any width: output frames use GStreamer’s default
BGRlayout, with each row padded to a multiple of 4 bytes, so widths that are not a multiple of 4 work with downstream elements such asvideoconvertand encoders.cam-proj requires calibration:
cam-projmode depends on theg3d/calibrationsticky event emitted byg3dobjectfuserorg3dbevfusion. Without a calibration for the selected camera the element falls back tobev.3D box convention: boxes are drawn as
GstAnalytics3DODMtddefines them:lengthalong the box’s own x axis,widthalong its y axis, rotated byyawcounter-clockwise about +z.Camera labels: each camera tile, and the
cam-projimage, is labelled with the camera’snamefrom theg3d/calibrationevent (g3dbevfusionsends nuScenes names such asCAM_FRONT), orCam <pad>without one.Box colours: a 3D box that
g3dobjectfuserassociated with a camera detection is white. Otherwise, when theg3d/calibrationevent listsclass-names(asg3dbevfusiondoes), boxes are coloured per class; if not, they are green.Point intensity: points are coloured by intensity, normalised to 0..1. Intensities above 1.5 in a frame are taken to be on a 0..255 scale.
cam-proj projects one camera at a time: When multiple camera streams are present, only the stream selected by
cam-proj-index(default0) is used as the projection background. Other camera streams are ignored in this mode.
Element Details (gst-inspect-1.0)#
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
application/x-lidar
multistream/x-analytics-batch(meta:GstAnalyticsBatchMeta)
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: BGR (gchararray)
width: [ 1, 32767 ] (GstIntRange)
height: [ 1, 32767 ] (GstIntRange)
framerate: [ 0/1, 120/1 ] (GstFractionRange)
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
cam-azimuth : Camera horizontal angle in degrees (perspective mode)
flags: readable, writable
Float. Range: -360 - 360 Default: 180
cam-distance : Camera distance from origin in meters (perspective mode)
flags: readable, writable
Float. Range: 1 - 500 Default: 35
cam-elevation : Camera elevation angle in degrees: 0=horizon 90=top-down (perspective mode)
flags: readable, writable
Float. Range: 5 - 89 Default: 30
cam-fov : Field of view in degrees (perspective mode)
flags: readable, writable
Float. Range: 10 - 150 Default: 60
forward-axis : Sensor axis that points in the vehicle's forward direction: x (KITTI) or y (nuScenes). bev mode draws it pointing up, and perspective mode measures cam-azimuth from it
flags: readable, writable
Enum "GstG3DRenderForwardAxis" Default: 0, "x"
(0): x - Sensor +x points forward (e.g. KITTI)
(1): y - Sensor +y points forward (e.g. nuScenes)
cam-proj-index : gvastreammux pad index of the camera to use for LiDAR projection in cam-proj mode; the first camera in the batch is used when no camera has this index
flags: readable, writable
Integer. Range: 0 - 255 Default: 0
cam-bg-dim : Brightness multiplier for the camera background in cam-proj mode (0.0=black, 1.0=original)
flags: readable, writable
Float. Range: 0 - 1 Default: 0.65
cam-bg-grayscale : Convert the camera background to grayscale in cam-proj mode
flags: readable, writable
Boolean. Default: true
height : Output image height in pixels
flags: readable, writable
Integer. Range: 1 - 32767 Default: 800
point-radius : Radius of each rendered point in pixels
flags: readable, writable
Integer. Range: 1 - 20 Default: 2
point-stride : Render every Nth point (1 = all points, 16 = every 16th point, etc.)
flags: readable, writable
Integer. Range: 1 - 100 Default: 16
view-mode : Rendering mode: bev (Bird's Eye View), perspective (3D perspective), or cam-proj (project LiDAR onto camera image using calibration)
flags: readable, writable
Enum "GstG3DRenderViewMode" Default: 0, "bev"
(0): bev - Bird's Eye View
(1): perspective - Perspective 3D
(2): cam-proj - Camera Projection
width : Output image width in pixels
flags: readable, writable
Integer. Range: 1 - 32767 Default: 800
zoom : BEV zoom factor: 1.0=default (50m range), 2.0=zoomed in (25m range), 0.5=zoomed out (100m range)
flags: readable, writable
Float. Range: 0.1 - 20 Default: 1