Skip to main content
Version: v1.2

EKS

Luna EKS Testing Tutorials

A few very basic test examples can be used to validate the functionality and operation of Luna.

ML Use-Cases /w NVIDIA gpu

By default, Luna will autoscale pods with the label elotl-luna=true. Kindly execute the following command kubectl apply -f followed by the provided YAML file. You will observe a pod running and completing its execution on a p3 instance equipped with an NVIDIA V100 GPU.

Upon completion of the pod, the corresponding node will be automatically terminated.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: gpu-test
annotations:
"node.elotl.co/instance-gpu-skus": "V100"
labels:
elotl-luna: "true"
spec:
restartPolicy: OnFailure
containers:
- name: cuda-vector-add
image: "registry.k8s.io/cuda-vector-add:v0.1"
resources:
limits:
nvidia.com/gpu: 1
EOF

You can monitor the creation of the new pod and node by running the command watch kubectl get pods,nodes. The test pod will only be active for a brief period after it starts. The GPU node that was added to support the pod will persist for a few more minutes.

To confirm the presence of a GPU on the node, you can run the kubectl describe node command and look for the "nvidia.com/gpu" entry or alternatively, you can run the following command:

kubectl get nodes "-o=custom-columns=NAME:.metadata.name,KUBELET STATUS:.status.conditions[3].reason,CREATED:metadata.creationTimestamp,VERSION:.status.nodeInfo.kubeletVersion,NVIDIA GPU(s):.status.allocatable.nvidia\.com/gpu"

General Testing (non-ML)

Luna will attempt to consolidate multiple smaller pods onto a newly deployed node, or in the case of larger pods, it will allocate a dedicated node for the pod, as occurs with the bin-selection packing mode.

You can perform simple testing using busybox or other pods of varying sizes. The following YAML files can be utilized, and the number of pods can be adjusted to observe Luna's dynamic response.

Small busybox deployment

Several busybox pods will be co-located within a single node

busybox

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox
spec:
replicas: 6
selector:
matchLabels:
app: busybox
template:
metadata:
labels:
app: busybox
elotl-luna: "true"
spec:
containers:
- name: busybox
image: busybox
resources:
requests:
cpu: 200m
memory: 128Mi
limits:
cpu: 300m
memory: 256Mi
command:
- sleep
- "infinity"
EOF

Larger busybox deployment

A single busybox pod will hit the threshold for bin-selection and be located on it's own node

busybox-large

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-large
spec:
replicas: 1
selector:
matchLabels:
app: busybox-large
template:
metadata:
labels:
app: busybox-large
elotl-luna: "true"
spec:
containers:
- name: busybox-large
image: busybox
resources:
requests:
cpu: 4
memory: 256Mi
limits:
cpu: 6
memory: 512Mi
command:
- sleep
- "infinity"
EOF

Check that the pods have started, the -o wide opton will show which does the pods are running on:

kubectl get pods -l elotl-luna=true -o wide

Sample output

NAME                             READY   STATUS    RESTARTS   AGE     IP                NODE                                            NOMINATED NODE   READINESS GATES
busybox-65cb45c86b-h9mz8 1/1 Running 0 3m40s 192.168.179.106 ip-192-168-180-235.us-east-2.compute.internal <none> <none>
busybox-65cb45c86b-ttgxz 1/1 Running 0 3m40s 192.168.180.96 ip-192-168-180-235.us-east-2.compute.internal <none> <none>
busybox-large-57f654fff8-nf4tj 1/1 Running 0 3m32s 192.168.181.40 ip-192-168-177-117.us-east-2.compute.internal <none> <none>

Next, we can verify the node information to confirm which instance types were selected and added to the Kubernetes cluster by Luna.

kubectl get nodes "-o=custom-columns=NAME:.metadata.name,KUBELET STATUS:.status.conditions[3].reason,CREATED:.metadata.creationTimestamp,VERSION:.status.nodeInfo.kubeletVersion,INSTANCE TYPE:.metadata.labels.node\.kubernetes\.io/instance-type,CPU(S):.status.capacity.cpu,MEMORY:.status.capacity.memory" --sort-by=metadata.creationTimestamp

Sample output

NAME                                            KUBELET STATUS   CREATED                VERSION               INSTANCE TYPE   CPU(S)   MEMORY
ip-192-168-62-44.us-east-2.compute.internal KubeletReady 2023-02-08T19:52:13Z v1.24.9-eks-49d8fe8 t3.xlarge 4 16202984Ki
ip-192-168-180-235.us-east-2.compute.internal KubeletReady 2023-02-10T16:40:11Z v1.24.9-eks-49d8fe8 t3a.xlarge 4 16248040Ki
ip-192-168-177-117.us-east-2.compute.internal KubeletReady 2023-02-10T16:40:26Z v1.24.9-eks-49d8fe8 t3a.2xlarge 8 32608568Ki

Zone Affinity and Spread Testing

Luna running on an EKS cluster supports kube-scheduler pod placement that includes zone spread or zone affinity. Luna recognizes zone spread expressed in the pod spec topologySpreadConstraints field as topologyKey set to topology.kubernetes.io/zone, and zone affinity expressed in the pod spec nodeAffinity field as the topology.kubernetes.io/zone in a zone value set. If Luna's placeNodeSelector option is set true, Luna also recognizes zone affinity expressed as a topology.kubernetes.io/zone nodeSelector. If Luna's placeBoundPVC option is set true, Luna also recognizes zone affinity if the pod has a persistent volume claim bound to a storage class or persistent volume with zone affinity. Currently Luna does not support both zone spread and zone affinity on the same pod; Luna webhook reports an error and skips the pod in this case.

If zone spread support is enabled for bin packing on EKS (option aws.binPackingZoneSpread=true), Luna creates at least one bin pack node in each of the cluster's zones, so that kube-scheduler has visibility into the full set of available zones. Then whenever Luna sees pending bin pack pod(s) with zone spread, it scales up the node count in each of the cluster's zones, to ensure that kube-scheduler can find needed zone-specific resources. Luna will scale down any unused nodes once the pending pods have been placed. When Luna sees pending bin pack pod(s) with zone affinity, it scales up the node count in an affine zone. And when Luna sees pending bin pack pod(s) with neither zone spread nor zone affinity, it scales up the node count in a single chosen zone.

Luna bin selection support for zone spread and affinity works similarly. By default (option reuseBinSelectNodes=true) Luna bin selection groups pods for node selection based on factors relevant to choosing a node type, including various pod spec fields and Luna annotations. For a pending pod in a group that includes zone spread, Luna scales up the node count in each of its zones. For a pending pod in a group that includes zone affinity, Luna scales up the node count in an affine zone, and for a pending pod in a group that includes neither zone spread nor zone affinity, it scales up the node count in a single chosen zone.

Examples are given below.

Small busybox zone affinity deployment

This small busybox pod will use Luna bin packing mode and is affine to the us-west-2d zone in a us-west-2 EKS cluster.

busybox-small-zone-affinity

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-small-zone-affinity
spec:
replicas: 1
selector:
matchLabels:
app: busybox-small-zone-affinity
template:
metadata:
labels:
app: busybox-small-zone-affinity
elotl-luna: "true"
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values:
- us-west-2d
containers:
- name: busybox-small-zone-affinity
image: busybox
resources:
requests:
cpu: 200m
memory: 128Mi
limits:
cpu: 300m
memory: 256Mi
command:
- sleep
- "infinity"
EOF

You can see the node running the zone affine pod:

kubectl get pods -l elotl-luna=true -l app=busybox-small-zone-affinity -o wide | awk {'print $1" " $7'} | column -t
NAME NODE
busybox-small-zone-affinity-65f957cd4-76hhq ip-192-168-10-108.us-west-2.compute.internal

And check the zone associated with that node to see it is us-west-2d:

kubectl get nodes -Ltopology.kubernetes.io/zone | awk {'print $1" " $6'} | grep gke-anne-regional-c2d-highcpu-4-e2c00890-kgwd
ip-192-168-10-108.us-west-2.compute.internal us-west-2d

Large busybox zone affinity deployment

This large busybox pod will use Luna bin selection mode and is affine to the us-west-2d zone in a us-west-2 EKS cluster.

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-affinity
spec:
replicas: 1
selector:
matchLabels:
app: busybox-affinity
template:
metadata:
labels:
app: busybox-affinity
elotl-luna: "true"
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: topology.kubernetes.io/zone
operator: In
values:
- us-west-2d
containers:
- name: busybox-affinity
image: busybox
resources:
requests:
cpu: 4
memory: 256Mi
limits:
cpu: 6
memory: 512Mi
command:
- sleep
- "infinity"
EOF

As in the previous example, you can see which node is running the zone affinity pod and can check that it is in the correct zone.

Small busybox zone spread deployment

These 3 small busybox pods will use Luna bin packing mode and be spread across the 3 zones in an EKS cluster.

busybox-small-zone-spread

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-small-zone-spread
spec:
replicas: 3
selector:
matchLabels:
app: busybox-small-zone-spread
template:
metadata:
labels:
app: busybox-small-zone-spread
elotl-luna: "true"
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: "topology.kubernetes.io/zone"
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: busybox-small-zone-spread
containers:
- name: busybox-small-zone-spread
image: busybox
resources:
requests:
cpu: 200m
memory: 128Mi
limits:
cpu: 300m
memory: 256Mi
command:
- sleep
- "infinity"
EOF

You can see the nodes running the zone spread pods:

kubectl get pods -l elotl-luna=true -l app=busybox-small-zone-spread -o wide | awk {'print $1" " $7'} | column -t
NAME NODE
busybox-small-zone-spread-5bd574fbdc-5pkpd ip-192-168-149-31.us-west-2.compute.internal
busybox-small-zone-spread-5bd574fbdc-dwkz2 ip-192-168-25-137.us-west-2.compute.internal
busybox-small-zone-spread-5bd574fbdc-w5z6q ip-192-168-174-122.us-west-2.compute.internal

And can check the zones associated with those nodes to see the spread:

kubectl get nodes -Ltopology.kubernetes.io/zone | awk {'print $1" " $6'} | column -t
NAME ZONE
ip-192-168-149-31.us-west-2.compute.internal us-west-2a
ip-192-168-174-122.us-west-2.compute.internal us-west-2c
ip-192-168-25-137.us-west-2.compute.internal us-west-2d
...

Large busybox zone spread deployment

These 3 large busybox pods will use Luna bin selection mode and be spread across the zones in an EKS cluster.

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
name: busybox-spread
spec:
replicas: 3
selector:
matchLabels:
app: busybox-spread
template:
metadata:
labels:
app: busybox-spread
elotl-luna: "true"
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: "topology.kubernetes.io/zone"
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: busybox-spread
containers:
- name: busybox-spread
image: busybox
resources:
requests:
cpu: 4
memory: 256Mi
limits:
cpu: 6
memory: 512Mi
command:
- sleep
- "infinity"
EOF

You can see the nodes running the zone spread pods:

kubectl get pods -l elotl-luna=true -l app=busybox-spread -o wide | awk {'print $1" " $7'} | column -t
NAME NODE
busybox-spread-77ccbd588-nb6h8 ip-192-168-171-139.us-west-2.compute.internal
busybox-spread-77ccbd588-vjr5t ip-192-168-21-145.us-west-2.compute.internal
busybox-spread-77ccbd588-zjr5z ip-192-168-139-212.us-west-2.compute.internal

And can check the zones associated with those nodes to see the spread:

kubectl get nodes -Ltopology.kubernetes.io/zone | awk {'print $1" " $6'} | column -t
NAME ZONE
ip-192-168-139-212.us-west-2.compute.internal us-west-2a
ip-192-168-171-139.us-west-2.compute.internal us-west-2c
ip-192-168-21-145.us-west-2.compute.internal us-west-2d
...