Skip to main content
Version: v0.5

Install Nova

This document describes how to install the Nova control plane and Nova agents.

Prerequisites

  1. Three Kubernetes clusters up and running. One cluster will be the hosting cluster where the Nova control plane will run. Two other clusters will be workload clusters that are managed by the Nova control plane.
  2. Packages: kubectl
  3. Nova cannot be deployed to an Autopilot GKE cluster. Please validate that you are deploying to a non-Autopilot cluster.
  4. Nova control plane uses etcd as a backing store, so hosting cluster should have storage provisioner.

Deploy

To deploy the Nova Control Plane to one cluster and to use another cluster as a workload cluster, make sure you have at least two contexts in your kubeconfig.

$ kubectl config get-contexts

CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* gke_elotl-dev_us-central1-c_nova-example-agent-1 gke_elotl-dev_us-central1-c_nova-example-agent-1 gke_elotl-dev_us-central1-c_nova-example-agent-1
gke_elotl-dev_us-central1-c_nova-example-agent-2 gke_elotl-dev_us-central1-c_nova-example-agent-2 gke_elotl-dev_us-central1-c_nova-example-agent-2
gke_elotl-dev_us-central1-c_nova-example-control-plane gke_elotl-dev_us-central1-c_nova-example-control-plane gke_elotl-dev_us-central1-c_nova-example-control-plane

Install Nova control plane

Note: The Nova control plane will be deployed to the elotl namespace, which will be created if it does not already exist.

To deploy Nova Control Plane to gke_elotl-dev_us-central1-c_nova-example-control-plane, run:

$ ./deploy_nova.sh gke_elotl-dev_us-central1-c_nova-example-control-plane ./nova-installer-output
----------------------------------------------------------
Nova Control Plane:
Kubernetes control plane is running at https://107.178.215.30

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
----------------------------------------------------------
Applying CRDs to Nova Control Plane...
customresourcedefinition.apiextensions.k8s.io/clusters.cluster.elotl.co configured

Nova installer script's output dir: ./nova-installer-output

Nova is deployed in kube-context:gke_elotl-dev_us-central1-c_nova-example-control-plane under namespace:elotl

To see Nova resources deployed to the hosting cluster, please run:
kubectl --context gke_elotl-dev_us-central1-c_nova-example-control-plane get all -n elotl

To see Nova logs, please run:
kubectl --context gke_elotl-dev_us-central1-c_nova-example-control-plane logs -n elotl -l component=nova-scheduler

To interact with Nova Control Plane, please run:
KUBECONFIG="./nova-installer-output/nova-kubeconfig" kubectl get all

To deploy Nova agent to a workload cluster, please run:
./deploy_agent.sh <workload-cluster-kube-context> <cluster-name>
where <cluster-name> is a unique cluster identifier in Nova Control Plane

Install Nova agent to the workload cluster

Each workload cluster needs a Nova agent. The Nova agent will be deployed to the elotl namespace, which will be created if it does not already exist.

To connect a workload cluster to Nova, we will deploy the Nova agent to gke_elotl-dev_us-central1-c_nova-example-agent-1 with "my-workload-cluster-1" identifier:

$ ./deploy_agent.sh gke_elotl-dev_us-central1-c_nova-example-agent-1 my-workload-cluster-1 ./nova-installer-output

If you have a second cluster, run the same command with a different cluster and cluster name, e.g.,

$ ./deploy_agent.sh gke_elotl-dev_us-central1-c_nova-example-agent-2 my-workload-cluster-2 ./nova-installer-output

Post installation check

We should now see the newly added workload cluster registered in Nova Control Plane:

$ KUBECONFIG="./nova-installer-output/nova-kubeconfig" kubectl get clusters
NAME                    K8S-VERSION   REGION        READY
my-workload-cluster-1 1.22 us-central1 True
my-workload-cluster-2 1.22 us-central1 True

To get more insight into the clusters available resources:

$ KUBECONFIG="./nova-installer-output/nova-kubeconfig" kubectl get clusters -o go-template-file=kubectl_templates/cluster_output.gotemplate


| CLUSTER NAME | K8S VERSION | CLOUD PROVIDER | REGION | STATUS |
|----------------------------------------------------------------------------------------------|
| my-workload-cluster-1 | 1.22 | gce | us-central1 | ClusterReady |
|----------------------------------------------------------------------------------------------|
| NODES |
|----------------------------------------------------------------------------------------------|
| NAME | AVAILABLE | AVAILABLE | AVAILABLE |
| | CPU | MEMORY | GPU |
| |
| gke-nova-example-agent-1-default-pool-25df6493-263w | 399m | 2332068Ki | 0 |
| gke-nova-example-agent-1-default-pool-25df6493-f9f8 | 427m | 2498615680 | 0 |
| |
| NODES' TAINTS |
| |
|----------------------------------------------------------------------------------------------|



| CLUSTER NAME | K8S VERSION | CLOUD PROVIDER | REGION | STATUS |
|----------------------------------------------------------------------------------------------|
| my-workload-cluster-2 | 1.22 | gce | us-central1 | ClusterReady |
|----------------------------------------------------------------------------------------------|
| NODES |
|----------------------------------------------------------------------------------------------|
| NAME | AVAILABLE | AVAILABLE | AVAILABLE |
| | CPU | MEMORY | GPU |
| |
| gke-nova-example-agent-2-default-pool-55fcf389-74zh | 457m | 2460060Ki | 0 |
| gke-nova-example-agent-2-default-pool-55fcf389-n77s | 359m | 2336086400 | 0 |
| gke-nova-example-agent-2-gpu-pool-950c3823-mlqq | 677m | 2354840Ki | 0 |
| |
| NODES' TAINTS |
| |
| gke-nova-example-agent-2-gpu-pool-950c3823-mlqq |
| - nvidia.com/gpu:present:NoSchedule |
|----------------------------------------------------------------------------------------------|

Uninstall Nova

Uninstall Nova control plane

To uninstall Nova control plane components, run:

$ kubectl --context <nova-control-plane-context> delete namespace <nova-control-plane-namespace>`.

Default namespace is elotl.

Uninstall Nova Agent from a workload cluster

$ `kubectl --context <nova-workload-context> delete namespace elotl`