How to perform WebRTC frame streaming#

DL Streamer Pipeline Server supports streaming the frames on WebRTC protocol using mediamtx media server. There is a dedicated docker compose file for demonstrating WebRTC streaming for DL Streamer Pipeline Server. It is available in DL Streamer Pipeline Server’s github repository, under the “docker” folder i.e., [WORKDIR]/edge-ai-libraries/microservices/dlstreamer-pipeline-server/docker/docker-compose-mediamtx.yml

Once a pipeline is started, DL Streamer Pipeline Server sends a stream of images through webrtc protocol to webrtc browser client. This is done via the MediaMTX server used for signaling. Note: As an optional recommendation, Coturn server can be used to facilitate NAT traversal and ensure that the webrtc stream is accessible on a non-native browser client and helps in cases where firewall is enabled. See example usage of coturn server in webrtc streaming here

Below are the necessary configuration to be aware of (or modify accordingly based on your deployment) in [WORKDIR]/edge-ai-libraries/microservices/dlstreamer-pipeline-server/docker/.env (They will be consumed appropriately in [WORKDIR]/edge-ai-libraries/microservices/dlstreamer-pipeline-server/docker/docker-compose-mediamtx.yml):

WHIP_SERVER_IP=<HOST_IP> # It should be the IP address of the machine on which open mediamtx container is running.
WHIP_SERVER_PORT=8889 # It is the port which is configured for mediamtx server. Default port is 8889.

After setting all the above information, we can start the WebRTC streaming:

Start the services

    docker compose -f docker-compose-mediamtx.yml up

Open another terminal and start a pipeline in DL Streamer Pipeline Server with the below curl command.

    curl http://localhost:8080/pipelines/user_defined_pipelines/pallet_defect_detection -X POST -H 'Content-Type: application/json' -d '{
    "source": {
        "uri": "file:///home/pipeline-server/resources/videos/warehouse.avi",
        "type": "uri"
    },
    "destination": {
        "metadata": {
            "type": "file",
            "path": "/tmp/results.jsonl",
            "format": "json-lines"
        },
        "frame": {
            "type": "webrtc",
            "peer-id": "pallet-defect-detection"
        }
    },
    "parameters": {
        "detection-properties": {
            "model": "/home/pipeline-server/resources/models/geti/pallet_defect_detection/deployment/Detection/model/model.xml",
            "device": "CPU"
        }
    }
}'

Open http://<HOST_IP>:8889/<peer-id> in your browser to view the WebRTC stream:

Note: If you are using 4K or high resolution video make sure to increase the bitrate to avoid choppy video streaming. You can set the bitrate by adding "bitrate" : 5000 with the webrtc configurations in your curl command.

  "frame": {
                  "type": "webrtc",
                  "peer-id": "pallet-defect-detection",
                  "bitrate": 5000
              }

Note: Mediamtx may fail to stream if the pipeline initialization takes longer than 10 seconds. To resolve this, you can increase the WHIP_SERVER_TIMEOUT value in the .env file located in the [WORKDIR]/edge-ai-libraries/microservices/dlstreamer-pipeline-server/docker/ directory.

How to perform WebRTC frame streaming#

This Page