Edge Orchestrator Upgrade Support#

Edge Orchestrator, including the Infrastructure Manager, is expected to support seamless upgrades between its consecutive versions starting from versions 3.0 and 3.1. To facilitate this, we provide mechanisms for both operators and developers:

Operators can perform a full upgrade of the Orchestrator in the production environment.
Developers can test modified infra charts to prevent breaking changes.

Upgradeability Requirements#

The following requirements must be met to consider the system upgradeable:

The Orchestrator operator can perform an in-place upgrade of the Edge Infrastructure Manager applications.
Any new version of OS Profiles is automatically populated, and end users can upgrade their configured Edge Nodes in place.
Downtime is allowed for individual components or services during upgrade activities.
The Orchestrator upgrade must not impact the operational state of existing Edge Nodes.
If the upgrade fails, manual recovery is possible through rollback procedures.

Full Upgrade of the Edge Orchestrator Platform in Production Environment#

Starting with version 3.1, operators running an earlier version of Edge Orchestrator are expected to be able to perform a seamless upgrade of the entire platform to the latest release. The Orchestrator upgrade guide provides the instructions for the operator to perform this upgrade.

Note

Automatic rollback to version 3.0 is not supported in Edge Orchestrator v3.1.

Upgrade Edge Manageability Applications in Validation Environment#

To prevent developers from introducing breaking changes in the Infrastructure Manager, all updates to the SB/NB APIs and database schema are validated during the CI phase through automated lint and integration checks.

To test the upgradeability of infra chart changes proposed in a pull request, we introduced the upgrade-test workflow. This workflow can be manually triggered on any pull request in the infra-charts repository, and is also executed periodically on the main branch to ensure continued compatibility.

In addition, developers can manually test an unreleased version of an Infrastructure Manager Helm chart in a locally deployed Edge Orchestrator. To do so, users can modify the chart revision of the following applications:

infra-managers
infra-core
infra-external
infra-onboarding

Using the platform’s web UI, developers can then verify the upgraded Orchestrator by:

Onboarding an Edge Node (EN)
Monitoring the EN’s status
Performing an EN upgrade

The following section provides step-by-step instructions for developers to upgrade the Edge Manageability applications and to verify that the upgrade meets the above requirements.

Prerequisites#

Edge Orchestrator v3.0 is deployed
You have obtained your JWT_TOKEN
A User, Organization, and Project have been created

Note

This guide assumes that your Orchestrator is deployed in a kind environment.

Register and Onboard an EN#

Perform EN registration and onboarding by following the steps in the EN registration guide. On the Host page of the Web UI, observe the status of the EN until it is fully provisioned and onboarded.

Upgrade Infra Manager Version#

To upgrade all Infrastructure Manager applications, log into your Edge Orchestrator and apply the following changes in the edge-manageability-framework repository cloned locally:

Add infra-charts repo to githubRepos in mage/Magefile.go:

sed -i '/var githubRepos = \[\]/a\    "https://github.com/open-edge-platform/infra-charts",' mage/Magefile.go

Get the commit hash of the infra-charts repository with the Helm charts you want to test and set INFRA_CHARTS_REVISION env variable accordingly.

Replace targetRevision with the current git hash of the infra-charts commit:

sed -i "s/targetRevision: [a-zA-Z0-9.]\+/targetRevision: ${INFRA_CHARTS_REVISION}/" argocd/applications/templates/infra-core.yaml
sed -i "s/targetRevision: [a-zA-Z0-9.]\+/targetRevision: ${INFRA_CHARTS_REVISION}/" argocd/applications/templates/infra-managers.yaml
sed -i "s/targetRevision: [a-zA-Z0-9.]\+/targetRevision: ${INFRA_CHARTS_REVISION}/" argocd/applications/templates/infra-onboarding.yaml
sed -i "s/targetRevision: [a-zA-Z0-9.]\+/targetRevision: ${INFRA_CHARTS_REVISION}/" argocd/applications/templates/infra-external.yaml

Replace chart: infra/charts/{{$appName}} with path: {{$appName}}:

sed -i 's|chart: infra/charts/{{$appName}}|path: {{$appName}}|g' argocd/applications/templates/infra-core.yaml
sed -i 's|chart: infra/charts/{{$appName}}|path: {{$appName}}|g' argocd/applications/templates/infra-managers.yaml
sed -i 's|chart: infra/charts/{{$appName}}|path: {{$appName}}|g' argocd/applications/templates/infra-onboarding.yaml
sed -i 's|chart: infra/charts/{{$appName}}|path: {{$appName}}|g' argocd/applications/templates/infra-external.yaml

Replace repoURL with git URL of infra-charts repository:

sed -i 's|repoURL: {{ required "A valid chartRepoURL entry required!" .Values.argo.chartRepoURL }}|repoURL: https://github.com/open-edge-platform/infra-charts.git|g' argocd/applications/templates/infra-core.yaml
sed -i 's|repoURL: {{ required "A valid chartRepoURL entry required!" .Values.argo.chartRepoURL }}|repoURL: https://github.com/open-edge-platform/infra-charts.git|g' argocd/applications/templates/infra-managers.yaml
sed -i 's|repoURL: {{ required "A valid chartRepoURL entry required!" .Values.argo.chartRepoURL }}|repoURL: https://github.com/open-edge-platform/infra-charts.git|g' argocd/applications/templates/infra-onboarding.yaml
sed -i 's|repoURL: {{ required "A valid chartRepoURL entry required!" .Values.argo.chartRepoURL }}|repoURL: https://github.com/open-edge-platform/infra-charts.git|g' argocd/applications/templates/infra-external.yaml

Commit the changes locally.

Apply updates to your deployment:

mage deploy:orchLocal dev
mage deploy:waitUntilComplete

Verify sync status of the applications:

kubectl -n dev get applications infra-core -o yaml
kubectl -n dev get applications infra-managers -o yaml
kubectl -n dev get applications infra-external -o yaml
kubectl -n dev get applications infra-onboarding -o yaml

Force Upgrade of Infra Manager Applications#

If changes to the Helm chart implementation do not result in modifications to the rendered Kubernetes manifests, the application update will not be triggered automatically. To force an update of the application, follow these steps:

Log into argo:
```
mage argo:login
```

Synchronize applications:

argocd app sync dev/infra-managers dev/infra-core dev/infra-external dev/infra-onboarding

Verify sync status of the applications:

kubectl -n dev get applications infra-core -o yaml
kubectl -n dev get applications infra-managers -o yaml
kubectl -n dev get applications infra-external -o yaml
kubectl -n dev get applications infra-onboarding -o yaml

Validate Upgraded Infrastructure Manager#

As part of the validation effort, developers should be able to verify that the Edge Orchestrator upgrade was successful. After the upgrade, all applications should appear in a Healthy state.

kubectl -n dev get applications

The functionality of the Edge Manager can be further validated by confirming it is able to discover new OS profiles and perform EN OS updates.

Verify OS Profiles#

After the Edge Manager upgrade, OS Profiles should still be automatically populated in the Web UI.

To verify if the OS profiles are automatically repopulated follow the steps:

Log into the web UI for the deployment.
Go to Settings and open the OS Profiles tab to see all OS profiles.

Identify the ResourceID of an unused profile:

CLUSTER_FQDN=edgeorchestration.example.com
PROJ_NAME=sample-project

curl -X GET \
   -H "Authorization: Bearer ${JWT_TOKEN}" \
   https://api.${CLUSTER_FQDN}/v1/projects/${PROJ_NAME}/compute/os

Use the OS profile’s ResourceID to delete an unused profile via an API call (e.g., RESOURCE_ID):

RESOURCE_ID=os-cd58ac2

curl -v -X DELETE "https://api.${CLUSTER_FQDN}/v1/projects/${PROJ_NAME}/compute/os/${RESOURCE_ID}" \
   -H "accept: application/json" \
   -H "Authorization: Bearer ${JWT_TOKEN}"

Soon, the deleted OS profile should be rediscovered by OS Resource Manager and displayed again in the Web UI. The OS profiles are updated by OS Resource Manager every 60 minutes.

Trigger EN Update#

An update of the onboarded EN OS will test if the EN and the Infrastructure Manager are still fully functional after the Edge Orchestrator upgrade.

Manually add an OS Profile using the latest EMT image details, for example:

OS_RESOURCES='{
   "architecture": "x86_64",
   "imageId": "3.0.20250514.2200",
   "imageUrl": "files-edge-orch/repository/microvisor/non_rt/edge-readonly-3.0.20250514.2200.raw.gz",
   "installedPackages": "",
   "kernelCommand": "",
   "name": "Edge Microvisor Toolkit 3.0.20250514",
   "osType": "OPERATING_SYSTEM_TYPE_IMMUTABLE",
   "profileName": "microvisor-nonrt",
   "repoUrl": "files-edge-orch/repository/microvisor/non_rt/edge-readonly-3.0.20250514.2200.raw.gz",
   "securityFeature": "SECURITY_FEATURE_NONE",
   "sha256": "f56ba0b338434813b70ecd8b81dec9a8f389a344ba868454825b4442217f6428",
   "updateSources": [],
   "osProvider": "OPERATING_SYSTEM_PROVIDER_INFRA"
}'

curl -X POST \
   -H "Accept: application/json" \
   -H "Authorization: Bearer ${JWT_TOKEN}" \
   --data "$OS_RESOURCES" \
   --header "Content-Type: application/json" \
   https://api.${CLUSTER_FQDN}/v1/projects/${PROJ_NAME}/compute/os

Identify your EN instance ResourceID in the list of instances and set INSTANCE environment variable accordingly:

curl -X GET \
   -H "Authorization: Bearer ${JWT_TOKEN}" \
   https://api.${CLUSTER_FQDN}/v1/projects/${PROJ_NAME}/compute/instances

Identify the OS Profile ResourceID in the list of profiles and set OSPROFILE environment variable accordingly.

curl -X GET \
   -H "Authorization: Bearer ${JWT_TOKEN}" \
   https://api.${CLUSTER_FQDN}/v1/projects/${PROJ_NAME}/compute/os

Update the EN instance’s desired_os to point to the OS profile:

curl -X PATCH \
   -H "Accept: application/json" \
   -H "Authorization: Bearer ${JWT_TOKEN}" \
   --data '{"osId":  "${OSPROFILE}"}' \
   --header "Content-Type: application/json" \
   https://api.${CLUSTER_FQDN}/v1/projects/${PROJ_NAME}/compute/instances/${INSTANCE}

Observe that the OS upgrade available note appears on the host page.
Schedule a maintenance window for your EN by following the guide on maintenance window creation.
Monitor the Update status to confirm the update was completed. The No new updates available status message indicates a successful update.