Skip to main content

Presto Cluster Autoscaling

Autoscaling helps you to easily manage your Presto clusters by adjusting the number of worker nodes automatically. Currently only auto scaling out based on CPU utilization is supported.

Overview#

When you create a new Presto cluster it can be challenging to know in advance the number of worker nodes that you need, and even more so to predict these requirements for the future. These number of worker nodes ideally should be sized to both ensure efficient performance and to avoid excess costs. Autoscaling can help achieve this balance by adjusting the number of worker nodes automatically as loads change over time, reducing the need for manual intervention.

There are currently two types of scaling strategy you can use to manage your cluster:

  • Static
  • Scale Out only (CPU)

A Static scaling strategy means that the number of worker nodes is constant while the cluster is being used.

A Scale Out only (CPU) scaling strategy means that the number of worker nodes is increasing based on the worker nodes average CPU utilization.

Additionaly, both scaling strategies have the ability to scale in to the minimum number of worker nodes when the cluster is idle for a user-specified amount of time.

Scaling Strategy

Static scaling strategy#

The Static scaling strategy is the default strategy when creating a Presto cluster.

When using this strategy, the defined number of worker nodes stay constant when the cluster is running. However the user has the ability to enable the scale to a single worker node when idle, which means that the cluster can automatically resize to a single worker node after no queries have been received by the cluster for a user-specified amount of time.

Static Scaling Strategy

Scale Out only scaling strategy#

The Scale Out only (CPU) scaling strategy gives the ability to define an auto-scaling policy for the Presto cluster. When first created the Presto cluster will start with the Minimum worker node count and if the average CPU utilization of these worker nodes goes above 75% then the number of worker nodes increase gradually to the Maximum worker node count by adding N number of worker nodes as defined in the Scale Out Step size.

However, if no queries have been received by the cluster for the period of time defined by the user, the cluster move to an Idle state, which triggers a resize to its Minimum worker node count.

Static Scaling Strategy

When does autoscaling occur?#

Autoscaling occurs when the average CPU utilization of the worker nodes goes above 75% for a period of 15mins.

The worker node count will increase until it reaches the Maximum worker node count.

When does idle time occur?#

Idle time occurs when there is no queries received by the cluster for a user-defined period of time, by default 30 mins

For a Static cluster the Idle time behaviour needs to be enabled, however for a Scale Out only (CPU) the Idle time scale in behaviour is enabled by default.

Restrictions and limitations#

  • Currently, a Presto cluster can only scale back in its minimum worker nodes count for Scale out only (CPU) scaling strategy or to a single worker node for the Static scaling strategy. A gradual scale in based on CPU utilization will be introduce in a subsequent release.

  • The Idle Time Window can only be between 10mins and 60mins. This will likely be increased in the future.

  • The time window for CPU utilization is not presently user configurable

Updating your autoscaling settings#

You can adjust the autoscaling settings easily by navigating to the cluster details view and clicking the Change Scaling Policy button.

Static Scaling Strategy

You can now edit The Minimum and Maximum Worker Node Count, as well as the Scale Out Step Size and the Time Window before scaling to minimum worker node count

Once you clicked the Save button, the new configuration will be applied to the Presto cluster.

Static Scaling Strategy