g3dlidarsrc#
Captures real-time point clouds from a physical LiDAR device and emits application/x-lidar buffers with attached LidarMeta. The element is a vendor-agnostic source: a single JSON configuration file selects the vendor SDK backend, the device model, and the transport (UDP today; USB and others reserved). Its output is byte-for-byte compatible with g3dlidarparse, so it can drive g3dinference and the rest of a 3D pipeline without any downstream changes.
Overview#
The g3dlidarsrc element is the live-capture counterpart to g3dlidarparse. Where g3dlidarparse replays recorded BIN/PCD frames from disk, g3dlidarsrc receives UDP packets directly from a sensor, decodes them via the vendor SDK, and converts the resulting frames into dense point-cloud buffers.
It is implemented as a GstPushSrc live source. The current backend is RoboSense (via the header-only rs_driver SDK), and the only hardware tested so far is the RSE1 (E1R). Other RoboSense models that rs_driver advertises support for (RS16, RS32, RS128, RSM1, etc.) are not validated here. The element has no compile-time dependency on any vendor SDK: each vendor is a separate backend shared library that the element loads at runtime via dlopen, selected by the config’s vendor field (see Vendor backends below).
Key operations:
Configuration-driven setup: A JSON
configfile declares thevendor,model, andtransport.Real-time capture and decode: A vendor SDK thread receives and decodes raw packets; completed frames are queued and handed to the pipeline.
Point-cloud conversion: Each frame is converted into a contiguous
float[x, y, z, intensity]payload, identical in layout tog3dlidarparseoutput.Metadata attachment: Emits
LidarMeta(point count, frame_id, timestamp, stream_id) on every buffer.Timeout detection: Fails the pipeline with a clear error if no data arrives within a configurable window.
Timestamp selection: Chooses between the LiDAR hardware clock and the GStreamer pipeline clock via a single
ntp-synctoggle.
Properties#
Property |
Type |
Description |
Default |
|---|---|---|---|
config |
String |
Path to a JSON config file describing the LiDAR vendor, model and transport. Required. |
null |
ntp-sync |
Boolean |
Synchronize received streams to the LiDAR clock (TRUE = use LiDAR clock, FALSE = use pipeline clock from 0) |
false |
timeout |
Uint64 |
Timeout in microseconds for receiving data (0 = no timeout) |
5000000 (5 s) |
stream-id |
Uint |
Stream identifier for this LiDAR source (used in metadata) |
0 |
Configuration#
The config property points to a JSON file with a deliberately layered structure so that adding a new vendor or a new transport is a localized change. There are three layers: vendor identity, transport (how data arrives, vendor-neutral), and an open passthrough object for vendor-specific extras.
A key principle: transport only carries fields whose meaning is vendor-neutral (for UDP today, that is just the bind address). Anything that uses vendor-specific terminology — for example RoboSense’s dense_points or wait_for_difop SDK options — belongs in params, not in transport.
Example#
{
"vendor": "robosense",
"model": "RSE1",
"transport": { "type": "udp", "bind_address": "192.168.1.100" },
"params": {
"msop_port": 6699,
"difop_port": 7788,
"max_distance": 80.0,
"dense_points": true
}
}
Top-level fields#
Field |
Required |
Description |
|---|---|---|
|
Yes |
Vendor identity; selects the SDK backend. Currently only |
|
Yes |
Device model within that vendor. |
|
Yes |
Object describing how data is transported (vendor-neutral). The |
|
No |
Free-form object for vendor/model-specific extras (port numbers, SDK options, …). It is not interpreted by the generic parser and is passed through to the corresponding vendor backend. |
UDP transport (transport.type = "udp")#
The UDP transport block contains only the fields that are meaningful to any UDP-based LiDAR vendor:
Field |
Required |
Default |
Description |
|---|---|---|---|
|
Yes |
— |
Transport discriminator. Must be |
|
No |
|
Local NIC IP to |
Reserved transports:
transport.typevalues other thanudp(for exampleusb) are recognized but not yet implemented. Such a config parses successfully but fails at pipeline start with an explicit error, rather than failing silently.
RoboSense params (vendor = "robosense")#
When vendor is robosense, the params object can carry the following keys to override rs_driver defaults. Every key is optional — omit it to keep the SDK default.
Keys are parsed and validated by the RoboSense backend on startup: any unrecognized key (typos included) fails the pipeline with an error listing every accepted key.
The fields below are validated on the only model we have tested (RSE1).
Key |
Group |
Type |
Default |
Description |
|---|---|---|---|---|
|
input |
int |
|
UDP port for MSOP packets (the point-cloud stream). Override only if your device has been reconfigured off its factory port. Must match the device’s web-UI setting. |
|
input |
int |
|
UDP port for DIFOP packets (device info, calibration, hardware clock — and on RSE1 the IMU payload). Same rule as above. |
|
input |
int |
|
Size of the kernel receive buffer ( |
|
decoder |
number |
|
Minimum point distance in metres. Points closer than this are dropped ( |
|
decoder |
number |
|
Maximum point distance in metres. Points beyond this are dropped ( |
|
decoder |
boolean |
|
If |
|
decoder |
boolean |
|
If |
|
decoder |
boolean |
|
If |
Pipeline Examples#
Basic capture pipeline#
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json ! fakesink
Capture with 3D inference and JSON export#
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json ! \
g3dinference config=pointpillars_ov_config.json device=CPU ! \
gvametaconvert add-tensor-data=true format=json json-indent=2 ! \
gvametapublish file-format=2 file-path=pointpillars.json ! \
fakesink
Timestamp source selection#
# Use the GStreamer pipeline clock (default; timestamps start from 0)
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json ntp-sync=false ! g3dinference ! ...
# Use the LiDAR hardware clock (absolute time, useful for multi-sensor fusion)
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json ntp-sync=true ! g3dinference ! ...
Timeout configuration#
# Default 5-second timeout
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json ! ...
# Shorter timeout for tests (3 seconds)
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json timeout=3000000 ! ...
# Disable the timeout (debugging)
gst-launch-1.0 g3dlidarsrc config=configs/robosense_e1r_udp.json timeout=0 ! ...
Input/Output#
Input Capability: none (this is a source element with no sink pad)
Output Capability:
application/x-lidar(buffer with attachedLidarMeta)
The output payload is a contiguous array of 32-bit floats, four per point in x, y, z, intensity order, repeated lidar_point_count times. This is the same layout g3dlidarparse produces, so g3dinference and other downstream elements accept either source unchanged.
Metadata#
The element attaches LidarMeta to each output buffer:
lidar_point_count: number of points in the frame (each point is 4 floats)frame_id: sequential frame counter, starting at 0 and incrementing per emitted frameexit_source_timestamp: timestamp when the buffer leaves the source, in nanoseconds (seentp-syncbelow)exit_g3dinference_timestamp: initialized toGST_CLOCK_TIME_NONE; filled in later byg3dinferenceif presentstream_id: the configuredstream-idproperty value
Timestamp behavior (ntp-sync)#
The ntp-sync property selects which clock the per-frame timestamp comes from:
|
Source clock |
Characteristics |
|---|---|---|
|
GStreamer pipeline clock |
Pipeline running time of the frame capture (starts from 0 in |
|
LiDAR hardware clock (from the per-MSOP-packet timestamp field) |
Absolute device time. Useful for multi-LiDAR fusion where sensors share a synchronized hardware clock. |
The chosen value is written to both LidarMeta.exit_source_timestamp and GST_BUFFER_PTS on every output buffer; GST_BUFFER_DTS is set to GST_CLOCK_TIME_NONE.
With ntp-sync=false the PTS marks when the frame was captured, not when it left the element. A frame is only complete after its whole scan period (e.g. 100 ms at 10 Hz) plus decode and queueing, so using the dequeue time would place the LiDAR roughly one frame period later than a camera frame captured at the same instant. The backend reports the capture time on the host wall clock (see the use_lidar_clock contract in g3d_lidar_backend_api.h) and the element converts it to running time. Which instant within the frame is reported depends on the backend (RoboSense: last point by default, first point with params.ts_first_point=true). If a backend reports a value that is not a recent host time, the element logs a warning once and falls back to the dequeue time.
The LiDAR starts streaming when the element starts, but frames are only pushed once the pipeline is in PLAYING, which can take a while if another source is slow to open its device. With ntp-sync=false, frames captured before the pipeline reached PLAYING have no running time, so the element drops them rather than pushing a burst of stale frames. The first buffer is the first frame captured after PLAYING.
Vendor backends#
g3dlidarsrc has no compile-time dependency on any vendor SDK. Each vendor is a separate backend shared library (libg3dlidar_<vendor>.so) that the element loads at runtime. At start() the element reads the config’s vendor field, derives the library name (vendor="robosense" → libg3dlidar_robosense.so), dlopens it, and resolves a small set of flat C functions via dlsym:
g3dlidarsrc (in libgst3delements.so) --dlopen("libg3dlidar_<vendor>.so")--> backend --> vendor SDK
(vendor-neutral, no SDK include) C ABI / function pointers (SDK + threading)
The C ABI is vendor-neutral and declared in g3d_lidar_backend_api.h (g3d_lidar_backend_create / set_callbacks / init / start / stop / destroy). It carries only fields common to all UDP LiDARs (model, bind address, timestamp policy); vendor-specific options ride in the raw params JSON object, which the element passes through verbatim and each backend parses itself. This is why the element does not hardcode a vendor allowlist and knows nothing about any vendor’s parameter schema — support for a vendor is simply “is its backend library present?”.
Backends live under g3dlidarsrc/backends/<vendor>/ and are each opt-in (default OFF), because building one pulls in that vendor’s SDK. Turning them all off still builds the element and plugin; g3dlidarsrc is registered and fails only at start() when the requested vendor’s backend is missing.
RoboSense backend (rs_driver)#
The RoboSense backend (backends/robosense/) wraps the header-only C++ rs_driver SDK — which has no prebuilt library and no stable C symbols of its own — behind the C ABI. It is built by the g3dlidar_robosense target, gated by the ENABLE_LIDAR_ROBOSENSE option (default OFF), producing libg3dlidar_robosense.so on Linux / g3dlidar_robosense.dll on Windows.
Linux#
# From the DL Streamer root
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DENABLE_LIDAR_ROBOSENSE=ON
cmake --build build --target gst3delements g3dlidar_robosense -j"$(nproc)"
Or with the top-level Makefile, which forwards the option:
make build ENABLE_LIDAR_ROBOSENSE=ON
Windows#
The all-in-one PowerShell script (see the Windows compilation guide) takes an -enableLidarRobosense switch, which forwards -DENABLE_LIDAR_ROBOSENSE=ON to CMake. Run it from an administrator PowerShell in the DL Streamer root:
# Build from source, including the RoboSense backend
.\scripts\build_dlstreamer_dlls.ps1 -enableLidarRobosense -setEnv
# Or build the installer package with the backend included (this is what CI runs)
.\scripts\build_dlstreamer_dlls.ps1 -buildInstaller -installerSkipCompression -enableLidarRobosense
Without -enableLidarRobosense the script builds everything except the backend (the CMake option stays OFF). The Windows CI already passes this switch, so the shipped installer includes g3dlidar_robosense.dll.
There are two ways to provide the rs_driver source it compiles against. If in doubt, use option B.
Option A — bring your own. Clone rs_driver yourself (matching the commit listed as
RS_DRIVER_VERSIONin the backend’s CMakeLists.txt), apply the in-tree patches once with the helper script, then point CMake at it via either-DRS_DRIVER_SRC_DIR=/path/to/rs_driver/srcor theRS_DRIVER_SRC_DIRenvironment variable. If the build complains about your tree (wrong commit, patches not applied, etc.), unset the variable and fall back to option B.bash <dlstreamer>/src/monolithic/gst/3d_elements/g3dlidarsrc/backends/robosense/patches/apply_patch.sh \ <dlstreamer>/src/monolithic/gst/3d_elements/g3dlidarsrc/backends/robosense/patches/*.patch
The helper is idempotent, so re-running it after a
git cleanis safe.Option B — let CMake do it (default). Don’t set
RS_DRIVER_SRC_DIR. The firstcmake ..withENABLE_LIDAR_ROBOSENSE=ONautomatically clones the pinned commit, applies the patches, and configures the build. One-time network download; nothing else to do.
Adding a new vendor#
Create
g3dlidarsrc/backends/<vendor>/with aCMakeLists.txtthat buildslibg3dlidar_<vendor>.so.Implement the functions in g3d_lidar_backend_api.h with C linkage, containing all of that vendor’s SDK code and threading. Parse the vendor’s own keys out of
params_json.Add an
option(ENABLE_LIDAR_<VENDOR> ... OFF)+add_subdirectory(<vendor>)in backends/CMakeLists.txt.
The element needs no changes — a config with "vendor": "<vendor>" will load the new backend automatically.
Runtime requirement#
Backends are loaded by bare name (libg3dlidar_<vendor>.so on Linux, g3dlidar_<vendor>.dll on Windows), so the install directory must be on the dynamic loader’s search path at runtime:
Linux — on
LD_LIBRARY_PATH. A standard install places the backend underlib/, whichscripts/setup_dls_env.shalready adds toLD_LIBRARY_PATH.Windows — on
PATH(Windows searchesPATH, notLD_LIBRARY_PATH). A standard install /setup_dls_env.ps1sets this up; building from the tree withbuild_dlstreamer_dlls.ps1 -setEnvalso adds the build output toPATH.
If the element fails at start with “Failed to load backend ‘…’”, the backend is either not built (its ENABLE_LIDAR_<VENDOR> option was OFF) or not on the library path.
Element Details (gst-inspect-1.0)#
Factory Details:
Rank none (0)
Long-name G3D LiDAR Source
Klass Source/Device
Description Receives real-time LiDAR point cloud data via rs_driver SDK (g3dlidarsrc)
Author Intel Corporation
Plugin Details:
Name 3delements
Description DL Streamer 3D Elements
Filename /home/kpi/dlstreamer/build/intel64/Release/lib/libgst3delements.so
Version 2026.2.0
License MIT/X11
Source module dlstreamer
Binary package Deep Learning Streamer elements
Origin URL https://github.com/open-edge-platform/dlstreamer/tree/main
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseSrc
+----GstPushSrc
+----GstG3DLidarSrc
Element Flags:
- SOURCE
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
application/x-lidar
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SRC: 'src'
Pad Template: 'src'
Element Properties:
automatic-eos : Automatically EOS when the segment is done
flags: readable, writable
Boolean. Default: true
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable, deprecated
Unsigned Integer. Range: 0 - 4294967295 Default: 0
config : Path to a JSON config file describing the LiDAR vendor, model and transport. Required.
flags: readable, writable
String. Default: null
do-timestamp : Apply current stream time to buffers
flags: readable, writable, deprecated
Boolean. Default: false
name : The name of the object
flags: readable, writable
String. Default: "g3dlidarsrc0"
ntp-sync : Timestamp source for buffer PTS (TRUE = LiDAR clock from DIFOP, FALSE = pipeline running time of the frame capture, starts from 0 at PLAYING)
flags: readable, writable
Boolean. Default: false
num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
stream-id : Stream identifier for this LiDAR source (used in metadata)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 0
timeout : Timeout in microseconds for receiving data (0 = no timeout)
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 5000000
typefind : Run typefind before negotiating (deprecated, non-functional)
flags: readable, writable, deprecated
Boolean. Default: false