Configure Image Pull Secret#

When applications are deployed to edge clusters, their containers or containerized VM images are always pulled from a registry.

There are two types of registries: public and private. Public registries are open and do not require any credentials for pulling images. However, an image pull secret is required for pulling images from a private registry.

This section describes how to configure the secret for a private registry.

Pulling Images from Common Private Registry#

In order to pull images, we need two steps:

Create Kubernetes secret
Set ImagePullSecret in Helm chart.

Create Kubernetes Secret#

First, you need to create a Kubernetes secret which includes the registry address and credentials in the Helm chart. You can follow the below link to create a Kubernetes secret: link.

Set ImagePullSecret#

After the secret is added, you need to set imagePullSecrets to the Kubernetes resources for containers and VMs deployment. For the container-based application, you can check and follow this link: ImagePullSecret for Container-based app

The VM-based application has a different format to set the ImagePullSecret from the container-based application. There are two examples below - one for VM-based application with ephemeral disk and the other one for VM-based application with PV disk.

For VMs with ephemeral disk (containerDisk)

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: <vm name>
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         image: <containerized VM image tag>
         imagePullSecret: <secret name>
         name: containerdisk

# example
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: ubuntu-vm
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         image: docker.io/exampleuser/ubuntu-vm:latest
         imagePullSecret: docker-secret-exampleuser
         name: containerdisk

For more information about the imagePullSecret for a VM application with ephemeral disk, check this document: ImagePullSecret for a VM-based application with ephemeral disk

For VMs with PV disk

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: <vm name>
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         dataVolume:
            name: containerdisk-dv
         name: containerdisk
   dataVolumeTemplates:
   - metadata:
      name: containerdisk-dv
   spec:
      source:
         registry:
         url: "docker://<registry_url>"
         secretRef: <secret name>
      pvc:
         accessModes:
         - ReadWriteOnce
         storageClassName: <PV class name>
         resources:
         requests:
            storage: <PV storage size>

# example
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: ubuntu-vm
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         dataVolume:
            name: containerdisk-dv
         name: containerdisk
   dataVolumeTemplates:
   - metadata:
      name: containerdisk-dv
   spec:
      source:
         registry:
         url: "docker://docker.io/exampleuser/ubuntu-vm:latest"
         secretRef: docker-secret-exampleuser
      pvc:
         accessModes:
         - ReadWriteOnce
         storageClassName: openebs-lvmpv
         resources:
         requests:
            storage: 100Gi

Note

url in registry should have docker:// prefix.

For more information about the imagePullSecret for a VM application with PV disk, check this document: ImagePullSecret for a VM-based application with PV disk

Pulling Images from Harbor Registry in Edge Orchestrator#

Unlike other private registries, the secret for the Harbor registry in Edge Orchestrator has already been created, which means that you do not need to create a secret in the Helm chart.

Edge Orchestrator provides an automated way to set the image pull secret to the Helm chart when deploying the Application. Here, you have to set the special string ‘%GeneratedDockerCredential%’ to Profile Chart Values. This feature requires two steps:

Make imagePullSecret(s) configurable at values.yaml file.
Override imagePullSecret(s) in values.yaml with special string %GeneratedDockerCredential% on Chart Values in Profile.

In case if the Helm chart uses Pre-hook templates which need images to be pulled from Harbor registry, then in addition to the above steps, you need to add this special line to the Profile Chart values:

Note

Note that the special string ‘%GeneratedDockerCredential%’ for the image pull secret MUST be set in Profile, not directly in Helm chart. If you set the special string directly to Helm chart, the Helm chart rendering will fail.

Make Image Pull Secret Configurable at values.yaml File#

An example for container and VM application deployment.

values.yaml file
```
...
image:
  pullSecret: ""
```

For Containers - Pod

apiVersion: v1
kind: Pod
metadata:
   name: <pod name>
spec:
   containers:
   - name: private-reg-container
   image: <your-private-image>
   imagePullSecrets:
   - name: {{ .Values.image.pullSecret }}

# example
apiVersion: v1
kind: Pod
metadata:
   name: nginx
spec:
   containers:
   - name: private-reg-container
   image: docker.io/exampleuser/nginx:latest
   imagePullSecrets:
   - name: {{ .Values.image.pullSecret }}

Note

imagePullSecrets is not only in Pod but other Kubernetes resources to deploy containers such as Deployment, StatefulSet, etc.

For VMs with ephemeral disk (containerDisk)

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: <vm name>
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         image: <containerized VM image tag>
         imagePullSecret: {{ .Values.image.pullSecret }}
         name: containerdisk

# example
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: ubuntu-vm
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         image: docker.io/exampleuser/ubuntu-vm:latest
         imagePullSecret: {{ .Values.image.pullSecret }}
         name: containerdisk

For VMs with PV disk

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: <vm name>
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         dataVolume:
            name: containerdisk-dv
         name: containerdisk
   dataVolumeTemplates:
   - metadata:
      name: containerdisk-dv
   spec:
      source:
         registry:
         url: "docker://<registry_url>"
         secretRef: {{ .Values.image.pullSecret }}
      pvc:
         accessModes:
         - ReadWriteOnce
         storageClassName: <PV class name>
         resources:
         requests:
            storage: <PV storage size>

# example
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: ubuntu-vm
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         dataVolume:
            name: containerdisk-dv
         name: containerdisk
   dataVolumeTemplates:
   - metadata:
      name: containerdisk-dv
   spec:
      source:
         registry:
         url: "docker://docker.io/exampleuser/ubuntu-vm:latest"
         secretRef: {{ .Values.image.pullSecret }}
      pvc:
         accessModes:
         - ReadWriteOnce
         storageClassName: openebs-lvmpv
         resources:
         requests:
            storage: 100Gi

Note

url in registry should have docker:// prefix.

Override Image Pull Secret in values.yaml With %GeneratedDockerCredential% on Chart Values in Profile#

Next, you should create or edit corresponding Applications’ Profile to override image pull secret with %GeneratedDockerCredential% in Chart Values. Following is an example:

image:
  pullSecret: '%GeneratedDockerCredential%'

Add Special Line to Allow Image Pull in Pre-Hook Template#

In case the helm chart has pre-hook templates which need to pull images from Harbor, then you should create or edit corresponding Applications’ Profile to override image pull secret with %GeneratedDockerCredential% in Chart Values and also add the additional line prehook: “%PreHookCredential%” in Profile chart values.

Following is an example:

image:
  pullSecret: '%GeneratedDockerCredential%'
  prehook: "%PreHookCredential%"

Inject Certificate Signed By Unknown Authority for VM-based Application With PV Disk#

If a containerized VM image is uploaded to the registry running with the certificate signed by unknown authority (ca.crt), you must add the certificate to the Helm chart. Otherwise, you can see the error message tls: failed to verify certificate: x509: certificate signed by unknown authority.

Note

This requires only when the VM-based Application will be running in PV disk, not ephemeral disk.

This is the step to add the certificate to Helm chart:

Get certificate

$ openssl s_client -showcerts -servername <registry_url> -connect <registry_url>:443 </dev/null

# example
$ openssl s_client -showcerts -servername registry.example.com -connect registry.example.com:443 </dev/null

Add ConfigMap to the Helm chart

apiVersion: v1
data:
   ca.crt: |-
   <Add certificate value here>
kind: ConfigMap
metadata:
   name: <cert configmap name>

# example
apiVersion: v1
data:
   ca.crt: |-
   -----BEGIN CERTIFICATE-----
   MIIIgTCCBmmgAwIBAgIQWUQSQfyDLEjAXhrG2HH7gTANBgkqhkiG9w0BAQsFADAp
   MQswCQYDVQQGEwJVUzEaMBgGA1UEChMRSW50ZWwgQ29ycG9yYXRpb24wHhcNMjQw
   NDI1MTUzMjUwWhcNMjUwNDI1MjAzMjUwWjApMQswCQYDVQQGEwJVUzEaMBgGA1UE
   ChMRSW50ZWwgQ29ycG9yYXRpb24wggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK
   AoICAQCcn4jZOY83fTNajHtx6Eq3yucTRsQCejBFuM4XKu9JQGuyHb6EgSB8WLF8
   -----END CERTIFICATE-----
kind: ConfigMap
metadata:
   name: vm-registry-tls-cert

Set certConfigMap in VirtualMachine

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: <vm name>
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         dataVolume:
            name: containerdisk-dv
         name: containerdisk
   dataVolumeTemplates:
   - metadata:
      name: containerdisk-dv
   spec:
      source:
         registry:
         url: "docker://<registry_url>"
         secretRef: {{ .Values.image.pullSecret }}
         certConfigMap: <cert configmap name>
      pvc:
         accessModes:
         - ReadWriteOnce
         storageClassName: <PV class name>
         resources:
         requests:
            storage: <PV storage size>

# example
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
   name: ubuntu-vm
spec:
   template:
   spec:
      domain:
         devices:
         disks:
         - disk:
               bus: virtio
            name: containerdisk
      volumes:
      - containerDisk:
         dataVolume:
            name: containerdisk-dv
         name: containerdisk
   dataVolumeTemplates:
   - metadata:
      name: containerdisk-dv
   spec:
      source:
         registry:
         url: "docker://docker.io/exampleuser/ubuntu-vm:latest"
         secretRef: {{ .Values.image.pullSecret }}
         certConfigMap: vm-registry-tls-cert
      pvc:
         accessModes:
         - ReadWriteOnce
         storageClassName: openebs-lvmpv
         resources:
         requests:
            storage: 100Gi

Note

url in registry should have docker:// prefix.

For more information, refer to: CDI TLS certificate config