How to deploy with Helm#

This guide provides step-by-step instructions for deploying the Document Summarization Sample Application using Helm.

Prerequisites#

Before you begin, ensure that you have the following prerequisites:

Steps to deploy with Helm#

Following steps should be followed to deploy Document Summarization application using Helm. You can install from source code or pull the chart from Docker hub.

Steps 1 to 3 varies depending on if the user prefers to build or pull the Helm details.

Option 1: Install from Docker Hub#

Step 1: Pull the Specific Chart#

Use the following command to pull the Helm chart from Docker Hub:

helm pull oci://registry-1.docker.io/intel/document-summarization --version <version-no>

🔍 Refer to the Docker Hub tags page for details on the latest version number to use for the sample application.

Refer to the release notes for details on the latest version number to use for the sample application.

Step 2: Extract the .tgz File#

After pulling the chart, extract the .tgz file:

tar -xvf document-summarization-<version-no>.tgz

This will create a directory named document-summarization containing the chart files. Navigate to the extracted directory.

cd document-summarization

Step 3: Configure the values.yaml File#

Edit the values.yaml file to set the necessary environment variables. Ensure you set the proxy settings as required.

Key

Description

Example Value

global.proxy.noProxy

NOPROXY

<your-no-proxy>

global.proxy.httpProxy

HTTP PROXY

<your-http-proxy>

global.proxy.httpsProxy

HTTPS PROXY

<your-https-proxy>

global.huggingface.token

Your Hugging Face API token

<your-huggingface-token

global.ui.nodePort

Sets the static port (in the 30000–32767 range)

global.otlp

OTLP Endpoint

<your-otlp-endpoint>

global.llm.llmModelId

Model to be used with ovms

Intel/neural-chat-7b-v3-3 or microsoft/Phi-3.5-mini-instruct

Option 2: Install from Source#

Step 1: Clone the Repository#

Clone the repository containing the Helm chart:

# Clone the latest on mainline
git clone https://github.com/open-edge-platform/edge-ai-libraries.git edge-ai-libraries
# Alternatively, Clone a specific release branch
git clone https://github.com/open-edge-platform/edge-ai-libraries.git edge-ai-libraries -b <release-tag>

Step 2: Change to the Chart Directory#

Navigate to the chart directory:

cd edge-ai-libraries/sample-applications/document-summarization/chart

Step 3: Configure the values.yaml File#

Edit the values.yaml file located in the chart directory to set the necessary environment variables. Refer to the table in Option 1, Step 3 for the list of keys and example values.

Step 4: Build Helm Dependencies#

Navigate to the chart directory and build the Helm dependencies using the following command:

helm dependency build

Common Steps after configuration#

Step 5: Deploy the Helm Chart#

Deploy the Document Summarization Helm chart:

helm install document-summarization . -n <your-namespace>

Note: When deploying OVMS, the OVMS service is observed to take more time than other model serving due to model conversion time.

Step 6: Verify the Deployment#

Check the status of the deployed resources to ensure everything is running correctly

kubectl get pods -n <your-namespace>
kubectl get services -n <your-namespace>

Step 7: Retrieving the Service Endpoint (NodePort and NodeIP)#

To access a docsum-nginx service running in your Kubernetes cluster using NodePort, you need to retrieve:

  • NodeIP – The internal IP of a worker node.

  • NodePort – The port exposed by the service.

Run the following command by replacing <namespace> with your actual values:

  # Step 1: List all the pods for the deployment and identify where docsum-nginx is running
  kubectl get pods -n <namespace> -o wide
  # Look for document-summarization-nginx pod to understand which node it is being deployed to.

  # Step 2: Get the node IP of the node where the service deployed to.
  kubectl get nodes -o wide
  # Look for the INTERNAL-IP value of the node deployed with the service

  # Step 3: Get the NodePort for the service deployed
  kubectl get svc document-summarization-nginx -n <namespace>
  # In the output, find the port listed as 80:<node-port>.
  # For example, in 80:30009/TCP, the NodePort is 30009.

  ## Step 4: Collect all the information above and paste it into your browser to access the service UI
  http://<node-ip>:<node-port>

Step 8: Update Helm Dependencies#

If any changes are made to the subcharts, update the Helm dependencies using the following command:

helm dependency update

Step 9: Uninstall Helm chart#

To uninstall helm charts deployed, use the following command:

helm uninstall document-summarization -n <your-namespace>

Verification#

  • Ensure that all pods are running and the services are accessible.

  • Access the application dashboard and verify that it is functioning as expected.

Troubleshooting#

  • If you encounter any issues during the deployment process, check the Kubernetes logs for errors:

    kubectl logs <pod-name>

  • If the PVC created during a Helm chart deployment is not removed or auto-deleted due to a deployment failure or being stuck, it must be deleted manually using the following commands:

    # List the PVCs present in the given namespace
kubectl get pvc -n <namespace>

# Delete the required PVC from the namespace
kubectl delete pvc <pvc-name> -n <namespace>