Usage of Custom OS Profile to Provision Edge Nodes#

This feature allows users to create and manage custom OS profiles that can be used to provision edge nodes. By defining a custom OS profile, users can specify custom built OS flavors of Ubuntu and Edge Microvisor Toolkit (EMT) as an edge node’s operating system. This enables a more tailored and efficient deployment process for edge nodes, allowing users to leverage their own OS images that are optimized for their specific use cases.

Prerequisites#

Here are the prerequisites to use the custom OS profile feature:

OS images must be built and hosted on a web server that is accessible during the edge node provisioning process.
Supported OS image formats include QCOW2 and RAW with or without compression. Supported compression formats are bzip2, bz2, gz, xz, and zst.
OS image including kernel must be signed with a trusted certificate to support Secure Boot, and the corresponding certificate must be enrolled in the edge node’s firmware.
OS images must include necessary drivers and configurations to support the target edge node hardware.
OS image partition layout must be labeled using GPT.
- UEFI/EFI system partition and rootfs partition for Ubuntu.
- Refer to the partition scheme for EMT OS.
OS image must have SHA256 checksum generated for verification during the provisioning process.
Users adding new OS profile to the EMF inventory using CLI must ensure there are no duplicate OS profile names and OS image version. Duplicate OS profile names and OS image versions can cause inconsistency during day 2 upgrade.
Ubuntu custom OS profiles must be mutable to allow for post-deployment configurations, updates, and installation of edge node agents.
EMT custom OS profiles must be immutable to ensure consistency and reliability across deployments. They should have edge node agents pre-installed to facilitate management and monitoring.
EMT custom OS profiles must have cloud-init package installed and configured to support automated initial setup during the provisioning process.
EMT custom OS profile must have EMF compatible version installed to ensure seamless integration with the EMF orchestrator.
EMT custom OS profiles must be based on the official EMT OS images provided by the Edge Microvisor Toolkit.
If an edge node is behind a proxy server, make sure the OS image URL is accessible during the provisioning process.
OS image URL must use HTTPS protocol to ensure secure transmission when hosting the OS image on the web server.

Default OS Profiles#

Default OS profiles available after the EMF orchestrator installation are:

Ubuntu 22.04 LTS
Ubuntu 24.04 LTS
Ubuntu 24.04 LTS with OOT GPU drivers
EMT 3.0 non-RT
EMT 3.0 RT
EMT standalone (used for OXM profile)
EMT with IDV (used for OXM profile)

Steps to Use the Custom OS Profile Feature#

Step 1: Get the TLS Certificate#

Get the TLS certificate from the server where the OS image is hosted. Replace <SERVER_DNS_NAME> (without https://) with your server’s DNS name.

Note

Make sure the image is hosted in raw.gz (EMT OS) or img (Ubuntu OS) format.

openssl s_client -connect <SERVER_DNS_NAME>:443 </dev/null 2>/dev/null | \
    openssl x509 -outform PEM > server_cert.pem
export TLS_CERT="$(base64 -w 0 server_cert.pem)"

Step 2: Create a Custom OS Profile YAML File#

Create the OS profile YAML file custom_os_profile.yaml with the following content:

appVersion: apps/v1
metadata:
  release: 0.0.0-dev
  version: 0.2.0
spec:
  name: <custom_os_profile_name>
  type: <IMMUTABLE/MUTABLE>
  provider: OS_PROVIDER_KIND_INFRA
  architecture: x86_64
  profileName: <custom_os_profile_name>
  image_url: <url_to_os_image>
  osImageVersion: <os_image_version>
  osImageSha256: <sha256_checksum_of_os_image>
  securityFeature: <SECURITY_FEATURE_NONE/SECURITY_FEATURE_SECURE_BOOT_AND_FULL_DISK_ENCRYPTION>
  description: "<description_of_custom_os_profile>"
  metadata: ""
  tlsCert: "<TLS_CERT>"

Replace the placeholders with appropriate values:

<custom_os_profile_name>: A unique name for the custom OS profile.
<IMMUTABLE/MUTABLE>: Specify whether the OS profile is immutable or mutable.
<url_to_os_image>: The URL where the OS image is hosted.
<sha256_checksum_of_os_image>: The SHA256 checksum of the OS image for verification.
<os_image_version>: The version of the OS image.
<SECURITY_FEATURE_NONE/SECURITY_FEATURE_SECURE_BOOT_AND_FULL_DISK_ENCRYPTION>: Specify the security features enabled for the OS profile.
<description_of_custom_os_profile>: A brief description of the custom OS profile.
<TLS_CERT>: The base64 encoded TLS certificate obtained in the previous step.

Optional: Skip Kernel Upgrade#

If you want to skip the kernel upgrade during the edge node provisioning, add the following to the metadata field in the YAML file:

appVersion: apps/v1
metadata:
  release: 0.0.0-dev
  version: 0.2.0
spec:
  name: <custom_os_profile_name>
  # ... other fields ...
  metadata: |
    skip_kernel_upgrade: "true"

Optional: Specify Kernel Version#

If you want to install a specific version of the Canonical kernel during the edge node provisioning, add the following to the metadata field in the YAML file:

Note

Replace <canonical_kernel_version> with the desired kernel version in the format linux-image-5.15.0-1035-generic. The kernel must be present in the Canonical repository.

Refer to the Ubuntu canonical package repository for available kernel versions.

appVersion: apps/v1
metadata:
  release: 0.0.0-dev
  version: 0.2.0
spec:
  name: <custom_os_profile_name>
  # ... other fields ...
  metadata: |
    kernelVersion: "<canonical_kernel_version>"

Step 3: Login to orch-cli#

Login to orch-cli first. Replace <EMF_KEYSTONE_URL>, <USERNAME>, and <PASSWORD> with your EMF orchestrator details:

orch-cli login <EMF_KEYSTONE_URL> --username <USERNAME> --password <PASSWORD>
orch-cli config set project <PROJECT_NAME>
orch-cli config set api-endpoint <EMF_API_ENDPOINT>

Replace the following placeholders:

<PROJECT_NAME>: Your project name.
<EMF_API_ENDPOINT>: Your EMF API endpoint (for example, https://api.<cluster_domain>).

Step 4: Create the Custom OS Profile#

Create the custom OS profile:

orch-cli create osprofile custom_os_profile.yaml

Step 5: Register the Edge Node#

Register the edge node with the created custom OS profile:

orch-cli create host --generate-csv

This command generates a CSV file test.csv with the necessary details to register the edge node.

Use the generated test.csv file to register the edge node with the custom OS profile. The OSProfile field in the CSV file should have the custom OS profile name created earlier.

orch-cli create host --import-from-csv test.csv

Step 6: Monitor the Provisioning Status#

Monitor the provisioning status of the edge node using the orch-cli or EMF web console:

orch-cli get host <host_resource_id>

Replace <host_resource_id> with the ID of the edge node obtained during registration.

Step 7: Verify the Installation#

Once the provisioning is complete, verify that the edge node is running the custom OS profile by checking the OS version and configurations on the edge node.

Step 8: Clean Up (Optional)#

Clean up the created custom OS profile if no longer needed:

orch-cli delete osprofile <custom_os_profile_name>

Replace <custom_os_profile_name> with the name of the custom OS profile created earlier.

Limitations#

1. Custom OS profile can used for only Day0 operation like edge node provisioning. Whereas Custom OS profile cannot be used for Day2 operations like OS upgrade using maintenance manager.

1. Operating system resource manager doesn’t manage the custom OS profiles hence it does not get the CVEs information of the operating systems