Enable MLOps#

Applications for industrial edge insights vision can also be used to demonstrate MLOps workflow using Model Download microservice. With this feature, during runtime, you can download a new model using the microservice and restart the pipeline with the new model.

To simplify this demonstration, we assume that models have already been downloaded to an accessible location (/tmp/models) using the Model Download from a running Geti server before restarting the pipeline.

Contents#

Pre-requisites#

We assume that Model Download service has already downloaded the model to be updated to /tmp/models. To learn how to setup Model Download, see here

If not available, you can simulate this by downloading the sample model from the edge-ai-resources repository. Once downloaded, extract to /tmp/models directory.

Steps#

Note: If you are running multiple instances of app, ensure to provide NGINX_HTTPS_PORT number in the url for the app instance, i.e., replace <HOST_IP> with <HOST_IP>:<NGINX_HTTPS_PORT>. If you are running a single instance and using an NGINX_HTTPS_PORT other than the default 443, replace <HOST_IP> with <HOST_IP>:<NGINX_HTTPS_PORT>.

Set up the sample application to start a pipeline. A pipeline named pallet_defect_detection_mlops is already provided in the pipeline-server-config.json for this demonstration with the pallet defect detection sample app.

Ensure that the pipeline inference element such as gvadetect/gvaclassify/gvainference should not have a model-instance-id property set. If set, this would not allow the new model to be run with the same value provided in the model-instance-id.

Navigate to the [WORKDIR]/edge-ai-suites/manufacturing-ai-suite/industrial-edge-insights-vision directory and set up the app.
```
cp .env_pallet-defect-detection .env
```

Update the following variables in .env file.

HOST_IP= # <IP Address of the host machine>

MINIO_ACCESS_KEY=   # MinIO service & client access key e.g. intel1234
MINIO_SECRET_KEY=   # MinIO service & client secret key e.g. intel1234

MTX_WEBRTCICESERVERS2_0_USERNAME=  # Webrtc-mediamtx username. e.g intel1234
MTX_WEBRTCICESERVERS2_0_PASSWORD=  # Webrtc-mediamtx password. e.g intel1234

Run the setup script using the following command.
```
./setup.sh
```
Bring up the containers
```
docker compose up -d
```
Check to see if the pipeline is present among the list of loaded pipelines which in our case is pallet_defect_detection_mlops.
```
./sample_list.sh
```

Modify the payload in apps/pallet-defect-detection/payload.json to launch an instance for the mlops pipeline.

[
  {
    "pipeline": "pallet_defect_detection_mlops",
    "payload": {
      "source": {
        "uri": "file:///home/pipeline-server/resources/videos/warehouse.avi",
        "type": "uri"
      },
      "destination": {
        "frame": {
          "type": "webrtc",
          "peer-id": "pdd"
        }
      },
      "parameters": {
        "detection-properties": {
          "model": "/home/pipeline-server/resources/models/pallet-defect-detection/deployment/Detection/model/model.xml",
          "device": "CPU"
        }
      }
    }
  }
]

Start the pipeline with the above payload.
```
./sample_start.sh -p pallet_defect_detection_mlops
```
Note the instance-id of the pipeline launched.
Verify the pipeline is running. You can View the WebRTC streaming on https://<HOST_IP>/mediamtx/<peer-str-id> by replacing <peer-str-id> with the value used in the original cURL command to start the pipeline.

Downloading a Model with Model Download

At this point, user would like to restart the pipeline with a newer model. The new model can be a retrained version of the existing model or a different model altogether. We use the Model Download microservice to help download the model. It supports downloading public models as well as Geti models from a running Geti server. To learn more about the microservice, see how to get started with it.

For our demonstration, we assume that:
- the Pallet Defect Detection model has been retrained and is available for download from a Geti server using the Model Download service.
- the downloaded location is accessible by the DL Streamer Pipeline Server. In our example, it is /tmp/models.
- the /tmp directory is already accessible by the sample application. If not, add it to the volumes section of dlstreamer-pipeline-server service in the docker-compose file.

Stop the running pipeline by using the pipeline instance “id”.

curl -k --location -X DELETE https://<HOST_IP>/api/pipelines/{instance_id}

Start a new pipeline with this new model. Before that modify the payload.json to use this new model in apps/pallet-defect-detection/payload.json. Notice the model path in the payload has changed to the new model.

[
  {
    "pipeline": "pallet_defect_detection_mlops",
    "payload": {
      "source": {
        "uri": "file:///home/pipeline-server/resources/videos/warehouse.avi",
        "type": "uri"
      },
      "destination": {
        "frame": {
          "type": "webrtc",
          "peer-id": "pdd"
        }
      },
      "parameters": {
        "detection-properties": {
          "model": "/tmp/models/pallet-defect-detection/deployment/Detection/model/model.xml",
          "device": "CPU"
        }
      }
    }
  }
]

Run the following.

./sample_start.sh -p pallet_defect_detection_mlops

View the WebRTC streaming on https://<HOST_IP>/mediamtx/<peer-str-id> by replacing <peer-str-id> with the value used in the original cURL command to start the pipeline.

Additional resources#

Downloading models from Geti Server#

To learn how to download models from a running Geti server, see here.

Note: The downloaded model(s) must be accessible to the DL Streamer Pipeline Server container. If necessary, add it to volumes section of dlstreamer-pipeline-server in compose file, and restart the DLSPS service.