GStreamer Python Bindings

1. GStreamer Python bindings

GStreamer provides Python bindings in the pygst module. On Ubuntu, you can install it through the apt package manager:

sudo apt-get install python3-gst-1.0

GStreamer Python interface has one-to-one mapping with C interface as Python wrappers are generated automatically via the GObject introspection. Similarly to C/C++, the Python application can construct a pipeline (either from a pipeline string description in the gst-launch format or creating and connecting elements programmatically), set the pad probe callback(s) on source or sink pad of any element in the pipeline, etc.

See the draw_face_attributes.py Python sample.

2. Video-analytics specific Python bindings

As GVA plugin registers inference specific metadata, another Python library - gstgva in this repository is complimentary to pygst and additionally provides Python bindings for GVA specific types such as GstGVATensorMeta and GstGVAJSONMeta and access to inference specific fields in GstVideoRegionOfInterestMeta.

Unlike gstpy, C to Python wrappers in gstgva are implemented via Python ctypes mechanism and designed with the goal of one-to-one mapping with GVA C++ interface.

You can find the following Python callback function example of accessing list of detected objects and objects attributes, very similar to the C++ function example on the Metadata page.

    def pad_probe_callback(pad, info):
        with gstgva.util.GST_PAD_PROBE_INFO_BUFFER(info) as buffer:
            caps = pad.get_current_caps()
            frame = gstgva.VideoFrame(buffer, caps)
            for roi in frame.regions():
                for tensor in roi.tensors():
                    print("  Attribute ", tensor.name())
                    print("    label=", tensor.label())
                    print("    confidence=", tensor.confidence())
        return Gst.PadProbeReturn.OK

3. gvapython element

In addition to Python bindings for standard GStreamer and GVA plugin interfaces, the gvapython GStreamer element from this repository could be used for pipeline customization with the code in Python language. The gvapython can be inserted at any place of the pipeline. It uses reference to a Python file and a function/class name as an element property, and invokes provided callback function on every frame.

The gvapython element could be used for post-processing of metadata generated by inference elements (for example, to extract list of detected bounding boxes, when the detection model format is not supported by GVA), or perform some analytics based on inference results (for example, to compare face embeddings versus gallery of known faces).

The gvapython element is implemented in “C” language as a normal GStreamer element and it invokes Python functions via “C” interface (Python.h).

See the samples for the gvapython element in face_detection_and_classification folder.

4. Performance considerations

Python code performance is typically significantly slower than C/C++ code compiled into a native instruction set. The parallelization is also limited as all Python code is executed in a single thread. If a Python callback contains compute intensive operations executed every frame, it may impact overall performance, and production usage may require migration from Python to C implementation.