Enable MLOps#

Applications for industrial edge insights vision can also be used to demonstrate MLOps workflow using Model Download microservice. With this feature, during runtime, you can download a new model using the microservice and restart the pipeline with the new model.

To simplify this demonstration, we assume that models have already been downloaded to an accessible location (/tmp/models) using the Model Download from a running Geti server before restarting the pipeline.

Contents#

Pre-requisites#

We assume that Model Download service has already downloaded the model to be updated to /tmp/models. To learn how to setup Model Download, see here

If not available, you can simulate this by downloading the sample model from edge-ai-resources repository here. Once downloaded, extract to /tmp/models directory.

Steps#

Note: If you are running multiple instances of app, ensure to provide NGINX_HTTPS_PORT number in the url for the app instance, i.e., replace <HOST_IP> with <HOST_IP>:<NGINX_HTTPS_PORT>. If you are running a single instance and using an NGINX_HTTPS_PORT other than the default 443, replace <HOST_IP> with <HOST_IP>:<NGINX_HTTPS_PORT>.

Set up the sample application to start a pipeline. A pipeline named pcb_anomaly_detection_mlops is already provided in the pipeline-server-config.json for this demonstration with the PCB anomaly detection sample app.

Ensure that the pipeline inference element such as gvadetect/gvaclassify/gvainference should not have a model-instance-id property set. If set, this would not allow the new model to be run with the same value provided in the model-instance-id.

Navigate to the [WORKDIR]/edge-ai-suites/manufacturing-ai-suite/industrial-edge-insights-vision directory and set up the app.
```
cp .env_pcb-anomaly-detection .env
```

Update the following variables in .env file.

HOST_IP= # <IP Address of the host machine>

MINIO_ACCESS_KEY=   # MinIO service & client access key e.g. intel1234
MINIO_SECRET_KEY=   # MinIO service & client secret key e.g. intel1234

MTX_WEBRTCICESERVERS2_0_USERNAME=  # Webrtc-mediamtx username. e.g intel1234
MTX_WEBRTCICESERVERS2_0_PASSWORD=  # Webrtc-mediamtx password. e.g intel1234

Run the setup script using the following command.
```
./setup.sh
```
Bring up the containers.
```
docker compose up -d
```
Check to see if the pipeline is loaded is present which in our case is pcb_anomaly_detection_mlops.
```
./sample_list.sh
```

Modify the payload in apps/pcb-anomaly-detection/payload.json to launch an instance for the mlops pipeline.

[
    {
        "pipeline": "pcb_anomaly_detection_mlops",
        "payload":{
            "source": {
                "uri": "file:///home/pipeline-server/resources/videos/anomalib_pcb_test.avi",
                "type": "uri"
            },
            "destination": {
            "frame": {
                "type": "webrtc",
                "peer-id": "anomaly"
            }
            },
            "parameters": {
                "classification-properties": {
                    "model": "/home/pipeline-server/resources/models/pcb-anomaly-detection/deployment/Anomaly classification/model/model.xml",
                    "device": "CPU"
                }
            }
        }
    }
]

Start the pipeline with the above payload.
```
./sample_start.sh -p pcb_anomaly_detection_mlops
```
Note the instance-id of the pipeline launched.
Verify the pipeline is running. You can View the WebRTC streaming on https://<HOST_IP>/mediamtx/<peer-str-id> by replacing <peer-str-id> with the value used in the original cURL command to start the pipeline.

Downloading a Model with Model Download

At this point, user would like to restart the pipeline with a newer model. The new model can bea retrained version of the existing model or a different model altogether. We use Model Download microservice to help download the model. It supports downloading public models as well as geti models from a running Geti server. To learn more about it, see here.

For our demonstration, we assume that:
- the PCB Anomaly Detection model has been retrained and is available for download from a Geti server using the Model Download service.
- the downloaded location is accessible by the DL Streamer Pipeline Server. In our example, it is /tmp/models.
- the /tmp directory is already accessible by the sample application. If not, add it to the volumes section of dlstreamer-pipeline-server service in the docker-compose file.

Stop the running pipeline by using the pipeline instance “id”.

curl -k --location -X DELETE https://<HOST_IP>/api/pipelines/{instance_id}

Start a new pipeline with this new model. Before that modify the payload.json to use this new model in apps/pcb-anomaly-detection/payload.json. Notice the model path in the payload has changed to the new model.

[
   {
      "pipeline": "pcb_anomaly_detection_mlops",
      "payload":{
            "source": {
               "uri": "file:///home/pipeline-server/resources/videos/anomalib_pcb_test.avi",
               "type": "uri"
            },
            "destination": {
            "frame": {
               "type": "webrtc",
               "peer-id": "anomaly"
            }
            },
            "parameters": {
               "classification-properties": {
                  "model": "/tmp/models/pcb-anomaly-detection/deployment/Anomaly classification/model/model.xml",
                  "device": "CPU"
               }
            }
      }
   }
]

Run the following.

./sample_start.sh -p pcb_anomaly_detection_mlops

View the WebRTC streaming on https://<HOST_IP>/mediamtx/<peer-str-id> by replacing <peer-str-id> with the value used in the original cURL command to start the pipeline.

Additional resources#

Downloading models from Geti Server#

To learn how to download models from a running Geti server, see here.

Note: The downloaded model(s) must be accessible to the DL Streamer Pipeline Server container. If necessary, add it to volumes section of dlstreamer-pipeline-server in compose file, and restart the DLSPS service.