Dynamic Vision

High performance 2D to 3D object classification and dynamic real-time space localization

Vision sub components:

The Vision container is a special process containing all the vision node in the same process, running on different threads, to allow zero copy memory operation on large data: RGB and Pointcloud streams.

The component of this container are:

• Realsense Camera node

• 2D AI perception. Yolo inference on OpenVINO™ provided as example

• 3D pointcloud alignment of the object in space

• Profiler for performance evaluation

The above high level diagram shows how the vision components interact each other:

• The Intel® RealSense™ camera node captures and publishes RGB and Point Cloud streams from the physical camera.

• The Object Detection node runs AI inference on the RGB stream and publishes the result for the Pose Detector.

• The Pose Detector subscribes to the Point Cloud stream and aligns the 3D data from the stream, with the storage mesh of corresponding classes in the subscribed bounding box. The Pose Detector then publishes the results in the RVC Messages’ API format.