Upcoming Features

This section lists some of the new features that will be coming soon to Nova.

Fill and Spill Scheduling

With the prevalence of AI/ML inference workloads on Kubernetes, there is a need to prioritize utilization of on-prem Kubernetes clusters over cloud K8s clusters. This allows teams to first leverage sunk-cost on-prem investments before utilizing pay-as-you-go cloud GPU resources.

Nova's new fill-and-spill scheduling policy will allow users to provide a prioritized (ordered) list of K8s clusters as targets for scheduling their workloads, where higher priority clusters will be preferred over lower priority ones.

Cost-aware Scheduling

Nova's cost-aware scheduling feature aims to allow users to automatically choose target clusters for their workloads based on resource costs associated with each cluster.

Given that compute is available at different price points in different regions on different cloud providers, cost-aware scheduling is beneficial in the following scenarios:

• Easily mobile workloads (like dev/test) where lowest price compute is all you care about
• Workloads that need expensive compute shapes (like GPU) where picking Cloud Provider A over Cloud Provider B would result in significant cost savings
• Batch workloads (such as data analytics, traditional AI/ML training jobs) where job completion times (or performance) is not a primary concern.

Sound interesting?

If any of these features are of interest to you, please write to us at info@elotl.co. We would love to learn more about your specific use-case and explore how best to help you.