Skip to main content
Version: v1.5

Upgrading

Key Concepts​

  • Nova Control Plane: The central management unit running on a hosting Kubernetes cluster.
  • Nova Agent: The component deployed to each workload cluster for management.
  • Workload Cluster: A Kubernetes cluster managed by the Nova Control Plane.
  • Hosting Cluster: A Kubernetes cluster where the Nova Control Plane runs.

Upgrade Control plane components​

  1. Download the tarball for the latest version and unpack it.
  2. Make sure there is a charts/ directory in the unpacked tarball.
  3. To upgrade your control plane components, run helm upgrade with the release name used at install time (e.g. nova-scheduler):
helm upgrade nova-scheduler charts/scheduler \
  --kube-context=${K8S_HOSTING_CLUSTER_CONTEXT} \
  -n ${NOVA_NAMESPACE}

Pass the same --set flags you used for helm install (image repositories, replica counts, registry credentials, JIT settings, etc.). 4. To upgrade the Nova CRDs, apply the nova_crds.yaml manifest to the Nova Control Plane cluster:

kubectl --context=${NOVA_CONTROLPLANE_CONTEXT} apply -f nova/install/base/crds/nova_crds.yaml --server-side
  1. Wait until control plane pods are up and running.

Breaking change: alert webhook authentication​

Starting with this release, the nova-scheduler /alert-webhook endpoint requires bearer token authentication. If you use the disaster recovery alert ingestion (Alertmanager → Nova ReceivedAlert), configure the token before or during the upgrade, otherwise alert ingestion returns 401 errors after the upgrade:

  1. Create a Secret: 
    kubectl --context=${K8S_HOSTING_CLUSTER_CONTEXT} -n ${NOVA_NAMESPACE} \
      create secret generic nova-alert-webhook-token \
      --from-literal=token='<long-random-token>'
  2. Pass it to the scheduler release: 
    helm upgrade nova-scheduler charts/scheduler ... \
      --set scheduler.alertWebhook.authTokenExistingSecret=nova-alert-webhook-token
  3. Give the same token to the alerting system, as described in the Recovery Framework.

See the Recovery Framework doc for novactl installs and details.

Note: if scheduler.alertWebhook.authTokenExistingSecret is set but the Secret (or its key) does not exist, the scheduler pod fails to start with CreateContainerConfigError, so the misconfiguration is visible immediately.

Note: the ReceivedAlert object naming scheme changed (name components are sanitized and disambiguated with a hash). A firing alert that already created a ReceivedAlert before the upgrade will create a second object after the upgrade, which re-triggers its RecoveryRun once. Recovery steps should be safe to re-run; plan accordingly if a step is not idempotent.

JIT Kind helper configuration​

The scheduler chart no longer sets KIND_HOST_ADDRESS by default. If you use the JIT Kind path (nova-jit-helper), configure it explicitly:

helm upgrade nova-scheduler charts/scheduler ... \
  --set scheduler.jit.kindHostAddress=172.17.0.1 \
  --set scheduler.jit.helperAuthTokenExistingSecret=nova-jit-helper-token

With novactl install control-plane, export NOVA_JIT_KIND_HOST_ADDRESS and NOVA_JIT_HELPER_AUTH_TOKEN_SECRET instead. Environment patching of the scheduler deployment still works but is reverted by the next helm upgrade.

Upgrade Nova agent in the workload cluster​

  1. Download the tarball for the latest version and unpack it.
  2. To upgrade your Nova agent, run helm upgrade with the release name (e.g. wlc-1):
helm upgrade wlc-1 charts/agent \
  --namespace ${NOVA_NAMESPACE} \
  --set clusterName=kind-workload-1