# GStreamer Python Bindings

## 1. GStreamer Python bindings

GStreamer provides Python bindings in the **pygst** module. On
Ubuntu, you can install it through the apt package manager:

```bash
sudo apt-get install python3-gst-1.0
```

GStreamer Python interface has one-to-one mapping with C interface as
Python wrappers are generated automatically via the GObject introspection.
Similarly to C/C++, the Python application can construct a pipeline (either
from a pipeline string description in the `gst-launch` format or creating and
connecting elements programmatically), set the pad probe callback(s) on
source or sink pad of any element in the pipeline, etc.

See the
[draw_face_attributes.py](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/libraries/dl-streamer/samples/gstreamer/python/draw_face_attributes/draw_face_attributes.py)
Python sample.

## 2. Video-analytics specific Python bindings

As GVA plugin registers inference specific metadata, another
Python library -
[gstgva](https://github.com/open-edge-platform/edge-ai-libraries/tree/main/libraries/dl-streamer/python/gstgva)
in this repository is complimentary to *pygst* and additionally provides
Python bindings for GVA specific types such as *GstGVATensorMeta* and
*GstGVAJSONMeta* and access to inference specific fields in
`GstVideoRegionOfInterestMeta`.

Unlike *gstpy*, C to Python wrappers in
*gstgva* are implemented via Python *ctypes* mechanism and designed with the
goal of one-to-one mapping with GVA C++ interface.

You can find the following Python callback function example of accessing
list of detected objects and objects attributes, very similar to the C++
function example on the [Metadata page](./metadata.md).

```bash
    def pad_probe_callback(pad, info):
        with gstgva.util.GST_PAD_PROBE_INFO_BUFFER(info) as buffer:
            caps = pad.get_current_caps()
            frame = gstgva.VideoFrame(buffer, caps)
            for roi in frame.regions():
                for tensor in roi.tensors():
                    print("  Attribute ", tensor.name())
                    print("    label=", tensor.label())
                    print("    confidence=", tensor.confidence())
        return Gst.PadProbeReturn.OK
```

## 3. gvapython element

In addition to Python bindings for standard GStreamer and GVA plugin interfaces,
the [gvapython](../elements/gvapython.md) GStreamer element from this repository
could be used for pipeline customization with the code in Python language.
The *gvapython* can be inserted at any place of the pipeline. It uses reference to
a Python file and a function/class name as an element property, and
invokes provided callback function on every frame.

The *gvapython* element could be used for post-processing of metadata
generated by inference elements (for example, to extract list of detected
bounding boxes, when the detection model format is not supported by GVA),
or perform some analytics based on inference results (for example, to
compare face embeddings versus gallery of known faces).

The *gvapython* element is implemented in "C" language as a normal GStreamer
element and it invokes Python functions via "C" interface (Python.h).

See the samples for the *gvapython* element in
[face_detection_and_classification](https://github.com/dlstreamer/dlstreamer/tree/master/samples/gstreamer/gst_launch/gvapython/face_detection_and_classification) folder.

## 4. Performance considerations

Python code performance is typically significantly slower than C/C++
code compiled into a native instruction set. The parallelization is also
limited as all Python code is executed in a single thread. If a Python callback
contains compute intensive operations executed every frame, it may
impact overall performance, and production usage may require migration
from Python to C implementation.