Skip to main content

Node pools and autoscaler

On a Connect plan, your apps run on your own nodes. Nodes are grouped in node pools: each pool has a node type (provider, instance type, boot disk), a minimum and a maximum number of nodes, and an autoscaler that adds and removes nodes between them.

Open Node pools in the user menu to see the pools of your private regions. Every member of the account sees them; an account admin can change them. You can also use the Node Pools API or the MCP tools list-node-pools, get-node-pool and update-node-pool-scaling.

How the autoscaler decides​

On every check, the autoscaler measures the pool's average CPU and memory usage (what the containers actually use, divided by what the pool's nodes make available to Kubernetes). The reservations shown on the Nodes page (what workloads request) do not drive scaling.

It adds a node when any of these is true:

  • a pod stays Pending longer than the pending pods grace period;
  • the pool CPU average is above the CPU target;
  • the pool memory average is above the memory target.

It removes one node only when all of these are true for the whole scale-down window:

  • the pool CPU average is below half of the CPU target;
  • the pool memory average is below half of the memory target;
  • no pod is Pending;
  • the pool has more nodes than its minimum.

Half of a target is rounded down: with a 75% memory target, memory must be below 37%. Each removal needs a new full window, so a pool shrinks one node at a time. The default window is 10 minutes.

The node removed is the one running the fewest pods among the nodes the autoscaler manages. Its pods move to the other nodes, then the VM is terminated or stopped, depending on the pool's scale-down action. Nodes marked Not managed by the autoscaler (for example, a node added to the pool from an existing VM) are never removed automatically.

Read the Autoscaler block​

Each pool shows an Autoscaler block with:

  • the last decision (scale up, scale down or no change), when it happened and its reason;
  • the pool CPU and memory averages, measured when you open the pool with the same calculation the autoscaler uses, next to the scale-up target and the scale-down threshold of each;
  • how long the pool has been below both thresholds, compared with the scale-down window.

If a pool is not shrinking, compare the two averages with their scale-down thresholds: both must be below. For each node, the pool also shows how many app pods run on it and the resources Kubernetes reserves there.

What you can change​

An account admin can change:

  • Scaling: minimum and maximum nodes, autoscale on or off, the CPU and memory targets, the pending pods grace period, the scale-down window and the scale-down action (terminate or stop the VM).
  • Power schedule: stop and start the pool on a schedule, or pin it running or stopped.
  • Stateful pools: stop or start one node, the data disk autogrow policy, and growing a data disk. Growing a disk cannot be undone. On OCI, the same action changes the disk performance (VPU per GB): new disks use 20 VPU/GB, and OCI bills each level on the provider account that owns the disk. AWS data disks (gp3) take their IOPS and throughput when they are created; both are shown on the disk, and AWS bills what is above the gp3 baseline (3000 IOPS, 125 MiB/s) on the provider account that owns the disk.

The node type, the network, the provider account and the cloud tags of a pool (the tags stamped on its VMs and disks at the provider) are managed by the Quave team. To change them, contact support.

Disk autogrow: node disk vs app volume​

There are two different disk policies:

  • Data disk autogrow (node pool): grows the cloud disk attached to a node of a stateful pool. Configure it on the pool.
  • Storage autoresize (app environment): grows the volume of one database or app environment inside that disk. Configure it on the environment, or with update-app-env-storage-autoresize.