Amazon EKS

This section details AWS specific configuration options.

Custom AMIs

NOTE: Non-bottlerocket (aws.isBottlerocketImage=false) custom AMIs must include the EKS nodes’ bootstrap script at /etc/eks/bootstrap.sh and bottlerocket (aws.isBottlerocketImage=true) custom AMIs must include aws.userData set to EKS cluster's TOML-formatted cluster information. Otherwise, nodes will not join the cluster.

You can tell Luna to use a specific AMI via the Helm values:

1. aws.amiIdGeneric for x86-64 nodes
2. aws.amiIdGenericArm for Arm64 nodes
3. aws.amiIdGpu for x86-64 nodes with GPU

Each of these configuration options accept an AMI ID. If the AMI doesn’t exist or is not accessible Luna will log an error and fall back to the latest generic EKS images.

Set these custom AMI IDs via helm values like this:

--set aws.amiIdGeneric=ami-1234567890
--set aws.amiIdGenericArm=ami-1234567890
--set aws.amiIdGpu=ami-1234567890

Custom AMIs with SSM

Amazon offers various EKS image families like Amazon Linux, Ubuntu, and BottleRocket. Luna can use AWS SSM to fetch the most up to date image from its store.

For Amazon Linux, you can get the latest EKS image for Kubernetes 1.27 on arm64 nodes at /aws/service/eks/optimized-ami/1.27/amazon-linux-2-arm64/recommended/image_id.

To configure a SSM query for each image types use imageSsmQueryGeneric, imageSsmQueryGenericArm, and imageSsmQueryGpu. All these parameters may include exactly one "%s" marker to replace with the Kubernetes version.

For example here’s how to use BottleRocket images:

--set aws.imageSsmQueryGeneric="/aws/service/bottlerocket/aws-k8s-%s/x86_64/latest/image_id"
--set aws.imageSsmQueryGenericArm="/aws/service/bottlerocket/aws-k8s-%s/arm64/latest/image_id"
--set aws.imageSsmQueryGpu="/aws/service/bottlerocket/aws-k8s-%s-nvidia/x86_64/latest/image_id"

To use Ubuntu:

--set aws.imageSsmQueryGeneric="/aws/service/canonical/ubuntu/eks/20.04/%s/stable/current/amd64/hvm/ebs-gp2/ami-id"
--set aws.imageSsmQueryGenericArm="/aws/service/canonical/ubuntu/eks/20.04/%s/stable/current/arm64/hvm/ebs-gp2/ami-id"

Block device mappings

To customize disk settings for your EKS nodes, use the aws.blockDeviceMappings option. Configure it with JSON with a format like this:

[
  {
    "DeviceName": "/dev/xvda",
    "Ebs": {
      "DeleteOnTermination": true,
      "VolumeSize": 42,
      "VolumeType": "gp2",
      "Encrypted": false
    }
  }
]

Use Helm’s --set-string, --set-json or --set-file options to set aws.blockDeviceMappings and avoid --set since it mangles its input.

For example:

$ cat block_device_mapping.json
[
  {
    "DeviceName": "/dev/xvda",
    "Ebs": {
      "DeleteOnTermination": true,
      "VolumeSize": 42,
      "VolumeType": "gp2",
      "Encrypted": false
    }
  }
]
$ helm ... --set-file aws.blockDeviceMappings=block_device_mapping.json

Bin Packing Zone Spread

When aws.binPackingZoneSpread is true (default false), Luna supports placement of bin packing pods that specify zone spread. To support bin packing zone spread, Luna keeps at least one bin packing node running in each zone associated with the EKS cluster as long as there are any Luna bin packing pods running.

User data

aws.userData allows you to define node setup information. The option is empty by default.

For non-bottlerocket images (aws.isBottlerocketImage=false), aws.userData specifies a script to be executed after nodes have been bootstrapped. For example specifying --set-string aws.userData="echo hello > /tmp/hello" will create a file named /tmp/hello with hello in it on the node once the EKS bootstrap script has completed. If you have a large script, we recommend you use the --set-file Helm option to load it:

$ cat myscript.sh
apt-get install my-package
$ ./deploy.sh ... --additional-helm-values "--set-file aws.userData=myscript.sh"

For bottlerocket images (aws.isBottlerocketImage=true), aws.userData specifies required EKS cluster information in TOML-format. This information can be generated into the file user-data.toml as described in https://github.com/bottlerocket-os/bottlerocket/blob/develop/QUICKSTART-EKS.md#kubernetes-cluster-info via:

eksctl get cluster --region your-cluster-region --name your-cluster-name -o json \
   | jq --raw-output '.[] | "settings.kubernetes.api-server = \"" + .Endpoint + "\"\nsettings.kubernetes.cluster-certificate =\"" + .CertificateAuthority.Data + "\"\n"' > user-data.toml

As with the non-bottlerocket case, you can use the --set-file Helm option to load it.

IMDS Metadata

metaData defines the instance metadata for EKS nodes, it’s a JSON document conforming to this specification.

Example:

{
    "HttpEndpoint": "enabled",
    "HttpProtocolIpv6": "disabled",
    "HttpPutResponseHopLimit": 42,
    "HttpTokens": "required",
    "InstanceMetadataTagsState": "enabled"
}

Default: Empty.

Use --set-string or --set-file with Helm to set the instance metadata, --set will mangle in the input.