Install Nova
This document describes how to install the Nova control plane and Nova agents.
Prerequisites
- Three Kubernetes clusters up and running. One cluster will be the hosting cluster where the Nova control plane will run. Two other clusters will be workload clusters that are managed by the Nova control plane.
- Packages:
kubectl - Nova cannot be deployed to an Autopilot GKE cluster. Please validate that you are deploying to a non-Autopilot cluster.
- Nova control plane uses etcd as a backing store, so hosting cluster should have storage provisioner.
Deploy
To deploy the Nova Control Plane to one cluster and to use another cluster as a workload cluster, make sure you have at least two contexts in your kubeconfig.
$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* gke_elotl-dev_us-central1-c_nova-example-agent-1 gke_elotl-dev_us-central1-c_nova-example-agent-1 gke_elotl-dev_us-central1-c_nova-example-agent-1
gke_elotl-dev_us-central1-c_nova-example-agent-2 gke_elotl-dev_us-central1-c_nova-example-agent-2 gke_elotl-dev_us-central1-c_nova-example-agent-2
gke_elotl-dev_us-central1-c_nova-example-control-plane gke_elotl-dev_us-central1-c_nova-example-control-plane gke_elotl-dev_us-central1-c_nova-example-control-plane
Install Nova control plane
Note: The Nova control plane will be deployed to the elotl namespace, which will be created if it does not already exist.
To deploy Nova Control Plane to gke_elotl-dev_us-central1-c_nova-example-control-plane, run:
$ ./deploy_nova.sh gke_elotl-dev_us-central1-c_nova-example-control-plane ./nova-installer-output
----------------------------------------------------------
Nova Control Plane:
Kubernetes control plane is running at https://107.178.215.30
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
----------------------------------------------------------
Applying CRDs to Nova Control Plane...
customresourcedefinition.apiextensions.k8s.io/clusters.cluster.elotl.co configured
Nova installer script's output dir: ./nova-installer-output
Nova is deployed in kube-context:gke_elotl-dev_us-central1-c_nova-example-control-plane under namespace:elotl
To see Nova resources deployed to the hosting cluster, please run:
kubectl --context gke_elotl-dev_us-central1-c_nova-example-control-plane get all -n elotl
To see Nova logs, please run:
kubectl --context gke_elotl-dev_us-central1-c_nova-example-control-plane logs -n elotl -l component=nova-scheduler
To interact with Nova Control Plane, please run:
KUBECONFIG="./nova-installer-output/nova-kubeconfig" kubectl get all
To deploy Nova agent to a workload cluster, please run:
./deploy_agent.sh <workload-cluster-kube-context> <cluster-name>
where <cluster-name> is a unique cluster identifier in Nova Control Plane
Install Nova agent to the workload cluster
Each workload cluster needs a Nova agent. The Nova agent will be deployed to the elotl namespace, which will be created if it does not already exist.
To connect a workload cluster to Nova, we will deploy the Nova agent to gke_elotl-dev_us-central1-c_nova-example-agent-1 with "my-workload-cluster-1" identifier:
$ ./deploy_agent.sh gke_elotl-dev_us-central1-c_nova-example-agent-1 my-workload-cluster-1 ./nova-installer-output
If you have a second cluster, run the same command with a different cluster and cluster name, e.g.,
$ ./deploy_agent.sh gke_elotl-dev_us-central1-c_nova-example-agent-2 my-workload-cluster-2 ./nova-installer-output
Post installation check
We should now see the newly added workload cluster registered in Nova Control Plane:
$ KUBECONFIG="./nova-installer-output/nova-kubeconfig" kubectl get clusters
NAME K8S-VERSION REGION READY
my-workload-cluster-1 1.22 us-central1 True
my-workload-cluster-2 1.22 us-central1 True
To get more insight into the clusters available resources:
$ KUBECONFIG="./nova-installer-output/nova-kubeconfig" kubectl get clusters -o go-template-file=kubectl_templates/cluster_output.gotemplate
| CLUSTER NAME | K8S VERSION | CLOUD PROVIDER | REGION | STATUS |
|----------------------------------------------------------------------------------------------|
| my-workload-cluster-1 | 1.22 | gce | us-central1 | ClusterReady |
|----------------------------------------------------------------------------------------------|
| NODES |
|----------------------------------------------------------------------------------------------|
| NAME | AVAILABLE | AVAILABLE | AVAILABLE |
| | CPU | MEMORY | GPU |
| |
| gke-nova-example-agent-1-default-pool-25df6493-263w | 399m | 2332068Ki | 0 |
| gke-nova-example-agent-1-default-pool-25df6493-f9f8 | 427m | 2498615680 | 0 |
| |
| NODES' TAINTS |
| |
|----------------------------------------------------------------------------------------------|
| CLUSTER NAME | K8S VERSION | CLOUD PROVIDER | REGION | STATUS |
|----------------------------------------------------------------------------------------------|
| my-workload-cluster-2 | 1.22 | gce | us-central1 | ClusterReady |
|----------------------------------------------------------------------------------------------|
| NODES |
|----------------------------------------------------------------------------------------------|
| NAME | AVAILABLE | AVAILABLE | AVAILABLE |
| | CPU | MEMORY | GPU |
| |
| gke-nova-example-agent-2-default-pool-55fcf389-74zh | 457m | 2460060Ki | 0 |
| gke-nova-example-agent-2-default-pool-55fcf389-n77s | 359m | 2336086400 | 0 |
| gke-nova-example-agent-2-gpu-pool-950c3823-mlqq | 677m | 2354840Ki | 0 |
| |
| NODES' TAINTS |
| |
| gke-nova-example-agent-2-gpu-pool-950c3823-mlqq |
| - nvidia.com/gpu:present:NoSchedule |
|----------------------------------------------------------------------------------------------|
Uninstall Nova
Uninstall Nova control plane
To uninstall Nova control plane components, run:
$ kubectl --context <nova-control-plane-context> delete namespace <nova-control-plane-namespace>`.
Default namespace is elotl.
Uninstall Nova Agent from a workload cluster
$ `kubectl --context <nova-workload-context> delete namespace elotl`