When traffic grows, an application may need more copies running at once. Those copies also need enough machines to run on. Kubernetes autoscaling addresses both jobs: a workload autoscaler adds application Pods, while a node autoscaler adds or removes the machines underneath them. A Pod is Kubernetes' unit for running one or more containers together.
The Horizontal Pod Autoscaler (HPA) adjusts the number of Pods using measurements such as CPU use. KEDA connects event sources, such as a queue of unfinished jobs, to scaling decisions. Tools such as Karpenter and Cluster Autoscaler deal with node capacity. These tools can work together: KEDA or HPA may request more workers, and a node autoscaler may then need to create space for them.
Choose the signal
CPU is useful for some services, but queue depth, concurrency or another application signal may better reflect demand. HPA adjusts replica counts from configured metrics. KEDA connects event sources to scaling behavior and is useful for event-driven workloads. Neither makes an application safe to stop or eliminates startup time.
Account for the full delay
- Measure how quickly the signal reflects demand.
- Include metric collection and scaling-controller response.
- Include scheduling, node provisioning and image download time.
- Include application initialization, dependency readiness and warm caches.
Set minimum warm capacity when that total delay exceeds the service's latency budget. Scaling to zero can reduce idle resources, but a request still needs somewhere to wait while capacity starts.
Node and workload constraints
Resource requests, affinity, topology, taints and available instance types determine whether new Pods fit. A node scaler cannot fix an impossible scheduling constraint. Resource requests that understate real consumption can make consolidation look successful until the node becomes overloaded.
Test shrinking as carefully as growing
Check graceful termination, work acknowledgement, PodDisruptionBudgets and connection draining. A worker terminated before it acknowledges a message can repeat work; terminating after an unsafe acknowledgement can lose work. Observe oscillation and deliberately unavailable metrics before trusting the loop.
This page explains the model without claiming a measured savings percentage. Compare EKS node operating models separately, and investigate consolidation blockers before relaxing disruption controls.
Which should you start with?
Start with HPA when an application metric reliably shows that the service needs more copies. Use KEDA when the amount of waiting work is the better signal and it supports your event source. Add node scaling when your cluster also needs to grow or shrink its machines. If new users cannot wait for a machine and application to start, keep some capacity running even during quiet periods.
Historical Kubedex content
Original publication: 2018-09-28T08:25:28+00:00. Preserved for context. Commands, versions, prices and results below reflect the original research.
Container platforms provide a magical experience for developers. Autoscaling stateless applications is one compelling feature that most developers love.
Kubernetes has had the ability to autoscale both pods and nodes for a while, however, these autoscaling features are quite basic.
A number of projects exist that improve how autoscaling works. In this blog we’ll compare the autoscalers that I’ve found. We’ll group them under two categories.
Cluster Autoscaling
Automatically adding new worker nodes to increase overall resources (cpu, memory, disk, network).
| Group | Name | Description |
|---|---|---|
| Node | Cluster Autoscaler | Scales nodes up and down in a Kubernetes cluster. Works on AWS, Azure and GCP. It scales nodes count by modifying the autoscaling group. In the background it constantly checks to see if pods on the cluster are failing to start because of insufficient resources and if they are it will trigger a scaling event. This autoscaler will wait until you have exhausted every resource and then begin to scale. It will also scale down when a certain threshold of resources are idle. |
| Node | Kubernetes EC2 Autoscaler | Node-level autoscaler for Kubernetes on AWS EC2 that is designed for batch jobs. The key metric used for scaling is the number of pending scheduled pods. This is more predictable if you know that spinning up a certain number of pods requires a certain number of nodes due to consistent resource usage on each. It also has a couple of nice features like AWS multi region support and the ability to safely drain nodes when scaling down. |
| Node | Kube AWS Autoscaler | Based on some frustration with the Cluster Autoscaler somebody decided to make a better one but specifically for AWS. One frustration is that the Cluster Autoscaler only scales when it’s too late i.e. resources are fully exhausted. It also doesn’t respect AZ placement so you could end up with an unbalanced cluster in terms of fault tolerance. The kube-aws-autoscaler also supports scaling multiple ASG’s. |
| Node | K8s AWS Autoscaler | AWS Kubernetes nodes autoscaler. |
| Node | Jupyterhub Autoscaler | JupyterHub Notebooks are used frequently by data scientists working across large amounts of data. This autoscaler will scale a cluster based on workload defined in a notebook. |
| Node | Cerebral | Kubernetes cluster autoscaler with pluggable metrics backends and scaling engines, as an alternative to the cluster-autoscaler project. |
| Node | Escalator | Escalator is a batch or job optimized horizontal autoscaler for Kubernetes. Ensures jobs have completed before terminating nodes and aims to scale clusters as quickly as possible to meet batch demands. AWS only currently. |
Pod Autoscaling
Horizontally increasing the number of pods or vertically increasing the resources a pod is given.
| Group | Name | Description |
|---|---|---|
| Pod | HPA | The Horizontal Pod Autoscaler (HPA) automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization (or, with custom metrics support, on some other application-provided metrics). |
| Pod | VPA | The Vertical Pod Autoscaler (VPA) automatically adjusts the amount of CPU and memory requested by pods running in the Kubernetes Cluster. Not to be used alongside HPA in a cluster (unless HPA is scaling on custom metrics only). |
| Pod | Addon Resizer | A simplified version of VPA that modifies resource requests of a deployment based on the number of nodes in the Kubernetes Cluster. |
| Pod | K8s Prom HPA | Before HPA supported custom metrics this was a great way to scale based on Prometheus monitoring data. |
| Pod | Cluster Proportional Autoscaler | Pods will proportionately increase or decrease with cluster size. Whereas with the HPA you must pre-configure CPU and memory on all containers, the cluster-proportional-autoscaler doesn’t need this as an input and will calculate it dynamically. |
| Pod | Kube AMQP Autoscaler | Scale pods based on AMQP message queue |
| Pod | K8s Rabbit Pod Autoscaler | Scale pods based on RabbitMQ messages |
| Pod | Kube SQS Autoscaler | Scale pods based on AWS SQS messages |
| Pod | Kube Cloudwatch Autoscaler | This is a Kubernetes deployment that will manage the autoscaling of one other Kubernetes deployment/replica/pod, periodically scaling the number of replicas based on any AWS CloudWatch metric (ex: SQS Queue Size or Max Age, ELB Response Time, etc). An example would be using it to increase the number of pods when the age of the oldest message on SQS gets too old, and decrease the number of pods when it stabilizes again. |
| Pod | Simple Scaler | The HPA has several shortcomings which can be listed as follows. The scaling is not restricted in terms of how pods are started and stopped at the same time. The scaling action is not based on historical data but on current usage. The thresholds for scaling up and scaling down are the same. This project aims to solve these issues. |
| Pod | Prometheus Kubernetes Autoscaler | Scale Kubernetes deployments based on Prometheus alerts. The autoscaler is registered as a webhook alertmanager receiver. When an alert is forwarded to this receiver the alertmanager will contact the autoscaler. |
| Pod | CronHPA | Cron Horizontal Pod Autoscaler(CronHPA) enables us to auto scale workloads (those that support scale subresource, e.g. deployment, statefulset) periodically using crontab scheme. |
Misc Autoscaling Apps
I found an autoscaler called the kubernetes-webhook-autoscaler this looks like a cool thing to use for really complex autoscaling tasks. You could build a simple API service outside of Kubernetes and abstract all of your cloud autoscaling logic behind it.
That’s it for my research so far. I’m keen to look at other autoscalers so if you find any please leave me a message or send me the details by the contact page.
The post Autoscaling appeared first on kubedex.com.
Historical workbook values; blanks mean unknown, not No. Prices, versions, maturity labels and feature claims are not current recommendations.
Kubernetes Autoscaling / Cluster (historical)
Historical snapshotRecovered comparison data. Versions, prices and availability describe the original research, not a current benchmark. Blank or damaged source values are marked unknown.
Choose columns
| Type | Name | Description |
|---|---|---|
| Cluster | Cluster Autoscaler | Scales nodes up and down in a Kubernetes cluster. Works on AWS, Azure and GCP. It scales nodes count by modifying the autoscaling group. In the background it constantly checks to see if pods on the cluster are failing to start because of insufficient resources and if they are it will trigger a scaling event. This autoscaler will wait until you have exhausted every resource and then begin to scale. It will also scale down when a certain threshold of resources are idle. |
| Cluster | Kubernetes EC2 Autoscaler | Node-level autoscaler for Kubernetes on AWS EC2 that is designed for batch jobs. The key metric used for scaling is the number of pending scheduled pods. This is more predictable if you know that spinning up a certain number of pods requires a certain number of nodes due to consistent resource usage on each. It also has a couple of nice features like AWS multi region support and the ability to safely drain nodes when scaling down. |
| Cluster | Kube AWS Autoscaler | Based on some frustration with the Cluster Autoscaler somebody decided to make a better one but specifically for AWS. One frustration is that the Cluster Autoscaler only scales when it’s too late i.e. resources are fully exhausted. It also doesn’t respect AZ placement so you could end up with an unbalanced cluster in terms of fault tolerance. The kube-aws-autoscaler also supports scaling multiple ASG’s. |
| Cluster | K8s AWS Autoscaler | AWS Kubernetes nodes autoscaler. |
| Cluster | Jupyterhub Autoscaler | JupyterHub Notebooks are used frequently by data scientists working across large amounts of data. This autoscaler will scale a cluster based on workload defined in a notebook. |
| Cluster | Cerebral | Kubernetes cluster autoscaler with pluggable metrics backends and scaling engines, as an alternative to the cluster-autoscaler project. |
| Cluster | Escalator | Escalator is a batch or job optimized horizontal autoscaler for Kubernetes. Ensures jobs have completed before terminating nodes and aims to scale clusters as quickly as possible to meet batch demands. AWS only currently. |
| Pod | HPA | The Horizontal Pod Autoscaler (HPA) automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization (or, with custom metrics support, on some other application-provided metrics). |
| Pod | VPA | The Vertical Pod Autoscaler (VPA) automatically adjusts the amount of CPU and memory requested by pods running in the Kubernetes Cluster. Not to be used alongside HPA in a cluster (unless HPA is scaling on custom metrics only). |
| Pod | Addon Resizer | A simplified version of VPA that modifies resource requests of a deployment based on the number of nodes in the Kubernetes Cluster. |
| Pod | K8s Prom HPA | Before HPA supported custom metrics this was a great way to scale based on Prometheus monitoring data. |
| Pod | Cluster Proportional Autoscaler | Pods will proportionately increase or decrease with cluster size. Whereas with the HPA you must pre-configure CPU and memory on all containers, the cluster-proportional-autoscaler doesn’t need this as an input and will calculate it dynamically. |
| Pod | Kube AMQP Autoscaler | Scale pods based on AMQP message queue |
| Pod | K8s Rabbit Pod Autoscaler | Scale pods based on RabbitMQ messages |
| Pod | Kube SQS Autoscaler | Scale pods based on AWS SQS messages |
| Pod | Kube Cloudwatch Autoscaler | This is a Kubernetes deployment that will manage the autoscaling of one other Kubernetes deployment/replica/pod, periodically scaling the number of replicas based on any AWS CloudWatch metric (ex: SQS Queue Size or Max Age, ELB Response Time, etc). An example would be using it to increase the number of pods when the age of the oldest message on SQS gets too old, and decrease the number of pods when it stabilizes again. |
| Pod | Simple Scaler | The HPA has several shortcomings which can be listed as follows. The scaling is not restricted in terms of how pods are started and stopped at the same time. The scaling action is not based on historical data but on current usage. The thresholds for scaling up and scaling down are the same. This project aims to solve these issues. |
| Pod | Prometheus Kubernetes Autoscaler | Scale Kubernetes deployments based on Prometheus alerts. The autoscaler is registered as a webhook alertmanager receiver. When an alert is forwarded to this receiver the alertmanager will contact the autoscaler. |
| Pod | CronHPA | Cron Horizontal Pod Autoscaler(CronHPA) enables us to auto scale workloads (those that support scale subresource, e.g. deployment, statefulset) periodically using crontab scheme. |
| Pod | KEDA | Scales pods based on Kafka, RabbitMQ or some Azure services. More scaling backends planned. |
20 rows
Sources & further reading
Spotted something that needs another look?
Help improve this page →