Skip to main content
Version: v1.2

OKE

Prerequisites

  1. Oracle bash cloud shell cli with the environment variable ENVSUBST pointing to an installation of envsubst (cloud shell does not allow root/sudo package installation).
  2. kubectl with correct context selected: pointing to the cluster you want to deploy Luna on.
  3. helm: the package manager for Kubernetes
  4. cmctl: the cert-manager command line utility
  5. An existing OKE cluster with at least 2 nodes (for Luna webhook replica availability) without autoscaling enabled.

Step 1(optional): Install Nvidia gpu driver for gpu workload

kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.13.0/nvidia-device-plugin.yml

Step 2: Deploy Luna

Luna needs cert-manager running in the cluster. The deploy script tries to detect cert-manager in the cluster and installs cert-manager to cert-manager namespace otherwise.

Luna supports 2 authentication methods for making OCI API calls: mounted OCI config file and instance principal.

The default authentication method uses an OCI config file mounted into the Luna node manager. To use this method, set up API keys for oracleidentitycloudservice/<account> and validate that access using ~/.oci/config with key ~/.oci/oci.pem works. Note that the key_file value specified inside the ~/.oci/config file should be expressed using the tilde path for the home directory, e.g., "key_file=~/.oci/oci.pem", so that it will work correctly when mounted in the pod. For this method, deploy using the following.

./deploy.sh --cluster-ocid <cluster-ocid> --config-file-full-path <config-file-full-path> --pem-file-full-path <pem-file-full-path> [--helm-release-name <release-name>] [--namespace <namespace>] [--additional-helm-values "<additional-helm-values>"]

To instead use the instance principal authentication method, set up an instance principal with sufficient service permissions. Information on configuring the instance principal is given in the section "Configuring an Instance Principal for Luna Use" below. For this method, deploy using the following:

./deploy.sh --cluster-ocid <cluster-ocid> --use-instance-principal [--helm-release-name <release-name>] [--namespace <namespace>] [--additional-helm-values "<additional-helm-values>"]

Note: The deploy.sh command generates a cluster-ocid_values.yaml file and a cluster-ocid_values_full.yaml file; please retain these files for use in future upgrades.

Step 3: Verify Luna

kubectl get all -n elotl

Sample Output
NAME                                      READY   STATUS    RESTARTS   AGE
pod/elotl-luna-manager-6bd7f4674d-cxwz6 1/1 Running 0 2m39s
pod/elotl-luna-webhook-7fcf5998b6-ltrd6 1/1 Running 0 2m39s
pod/elotl-luna-webhook-7fcf5998b6-svr6b 1/1 Running 0 2m39s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/elotl-luna-manager ClusterIP x.x.x.x <none> 9090/TCP 2m39s
service/elotl-luna-webhook ClusterIP x.x.x.x <none> 8443/TCP 2m39s

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/elotl-luna-manager 1/1 1 1 2m39s
deployment.apps/elotl-luna-webhook 2/2 2 2 2m39s

NAME DESIRED CURRENT READY AGE
replicaset.apps/elotl-luna-manager-6bd7f4674d 1 1 1 2m39s
replicaset.apps/elotl-luna-webhook-7fcf5998b6 2 2 2 2m39s

Step 4: Run some workloads!

Follow our tutorial to understand value provided by Luna.

Step 5: Verify test pod launch and dynamic worker node addition/removal (while testing)

kubectl get pods --selector=elotl-luna=true -o wide -w
kubectl get nodes -w

Upgrade

When running the upgrade command described below, set <retained-values-file> to <retained-path>/<cluster-name>_values_full.yaml, if your installation version was post 0.5.4 Luna, and to <retained-path>/<cluster-name>_values.yaml otherwise.

To upgrade an existing luna deployment, run:

helm upgrade elotl-luna <chart-path> --wait --namespace=<cluster-namespace> --values=<retained-values-file> <additional-helm-values(optional)>

For example, to upgrade my-cluster from luna-v0.4.6 to luna-v0.5.0 and set an additional helm value binPackingNodeCpu=2, run:

helm upgrade elotl-luna ./elotl-luna-v0.5.0.tgz --wait --namespace=elotl --values=../../luna-v0.4.6/oke/ocid1.cluster.oc1.iad.aaaaaaaasxamzqh2ch6fmxvk5uierz6sxyicmutae2b2e25tvcybh2azdukq_values.yaml --set binPackingNodeCpu=2

And validate the upgrade as follows:

helm ls -A
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
elotl-luna elotl 4 2023-05-19 14:15:30.686251 -0700 PDT deployed elotl-luna-v0.5.0 v0.5.0

Cleanup

helm uninstall elotl-luna --namespace=elotl
kubectl delete namespace elotl

Configuring an Instance Principal for Luna Use

There are various approaches to configuring an instance principal that will allow Luna to manage compute resources in your cluster. General background information is given here.

One approach is as follows:

  • Create a compartment-level dynamic group in the default domain containing the statically-allocated nodes that may host Luna in the cluster, e.g., create luna-dyn-group with the rule:

    Any {instance.id = 'ocid1.instance.oc1...', instance.id = 'ocid1.instance.oc1...'}

  • Create a tenancy-level policy to allow the nodes in that rule to manage OCI resources, e.g.:

    Allow dynamic-group luna-dyn-group to manage all-resources in tenancy