GKE
Prerequisites
- gcloud cli
- kubectl with correct context selected: pointing to the cluster you want to deploy Luna on.
- helm: the package manager for Kubernetes
- cmctl: the cert-manager command line utility
- An existing zonal or regional GKE standard (not Autopilot) cluster with at least 2 nodes (for Luna webhook replica availability).
Step 1(optional): Install Nvidia gpu driver for gpu workloads
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/master/nvidia-driver-installer/cos/daemonset-preloaded.yaml
Step 2: Deploy Luna
Luna needs cert-manager running in the cluster. Deploy script tries to detect cert-manager in the cluster and installs cert-manager to cert-manager namespace otherwise.
For the deploy.sh script, the cluster-location argument should be set to the appropriate zone for a zonal GKE cluster (e.g., us-central1-a) or the broader region for a regional GKE cluster (e.g., us-central1). By default, the script will use the namespace "elotl" and the release name "elotl-luna" for deployment. However, you can override these defaults using the --namespace and --helm-release-name options, respectively.
Before using Luna, make sure workload identity is enabled on the cluster and its specified node pool. In case workload identity is not enabled, the deployment script will ask for your confirmation to enable it during deployment. Alternatively, you may use the --enable-workload-identity option to enable it directly without any prompts.
./deploy.sh --name <cluster-name> --project <project-id> (--zone <compute-zone> or --region <compute-region>) --node-pool <node-pool> [--helm-release-name <release-name>] [--namespace <namespace>] [--enable-workload-identity] [--additional-helm-values "<additional-helm-values>"]
Note: This command generates a gke-cluster-name_values.yaml file and (in post 0.6.0 Luna) a gke-cluster-name_helm-release-name_values_full.yaml file; please retain these files for use in future upgrades.
Step 3: Verify Luna
kubectl get all -n elotl
Sample Output
NAME READY STATUS RESTARTS AGE
pod/luna-manager-5d8578565d-86jwc 1/1 Running 0 56s
pod/luna-webhook-58b7b5dcfb-dwpcb 1/1 Running 0 56s
pod/luna-webhook-58b7b5dcfb-xmlds 1/1 Running 0 56s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/luna-webhook ClusterIP x.x.x.x <none> 8443/TCP 57s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/luna-manager 1/1 1 1 57s
deployment.apps/luna-webhook 2/2 2 2 57s
NAME DESIRED CURRENT READY AGE
replicaset.apps/luna-manager-5d8578565d 1 1 1 57s
replicaset.apps/luna-webhook-58b7b5dcfb 2 2 2 57s
Step 4: Run some workloads!
Follow our tutorial to understand value provided by Luna.
Step 5: Verify test pod launch and dynamic worker node addition/removal (while testing)
kubectl get pods --selector=elotl-luna=true -o wide -w
kubectl get nodes -w
Upgrade
When running the upgrade command described below, set <retained-values-file> to <retained-path>/<cluster-name>_values_full.yaml, if your installation version was post 0.5.4 Luna, and to <retained-path>/<cluster-name>_values.yaml otherwise.
To upgrade an existing luna deployment, run:
helm upgrade elotl-luna <chart-path> --wait --namespace=<cluster-namespace> --values=<retained-values-file> <additional-helm-values(optional)>
For example, to upgrade my-cluster from luna-v0.5.3 to luna-v0.5.4 and set an additional helm value binPackingNodeCpu=2, run:
helm upgrade elotl-luna ./elotl-luna-v0.5.4.tgz --wait --namespace=elotl --values=../../luna-v0.5.3/gke/my-cluster_values.yaml --set binPackingNodeCpu=2
And validate the upgrade as follows:
helm ls -A
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
elotl-luna elotl 1 2023-07-26 14:15:30.686251 -0700 PDT deployed elotl-luna-v0.5.4 v0.5.4
Cleanup
helm uninstall elotl-luna --namespace=elotl
kubectl delete namespace elotl
gcloud container node-pools delete luna-nodepool --zone=$COMPUTE_REGION --cluster=$CLUSTER_NAME