As a platform administrator, you can configure a node upgrade strategy to tune how GKE upgrades the nodes in your Google Kubernetes Engine (GKE) clusters. To learn more about node upgrade strategies, see Node upgrade strategies.
Before you beginBefore you start, make sure that you have performed the following tasks:
gcloud components update
. Note: For existing gcloud CLI installations, make sure to set the compute/region
property. If you use primarily zonal clusters, set the compute/zone
instead. By setting a default location, you can avoid errors in the gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location
. You might need to specify the location in certain commands if the location of your cluster differs from the default that you set.When configuring your cluster's node pools, you can select and configure one of the supported node upgrade strategies, namely surge or blue-green. Using these upgrade strategies lets you optimize the node pool upgrade process based on your cluster environment's needs.
Note: By default, node pools use the surge upgrade strategy withmaxSurge=1
and maxUnavailable=0
. Configure surge upgrades
Surge upgrades allow you to change the number of nodes GKE upgrades at one time and the amount of disruption an upgrade makes on your workloads.
The max-surge-upgrade
and max-unavailable-upgrade
flags are defined for each node pool. For more information on choosing the right parameters, go to Optimize your surge upgrade configuration.
maxSurge=1 maxUnavailable=0
. To tune your configuration, see Optimize your surge upgrade configuration. Warning: For nodes that use Spot VMs, surge upgrade values are ignored because there is no availability guarantee. During an upgrade, old nodes are drained directly without waiting for surge nodes that use Spot VMs to be ready.
You can change these settings when creating or updating a cluster or node pool.
The following variables are used in the commands mentioned below:
CLUSTER_NAME
: the name of the cluster for the node pool.COMPUTE_ZONE
: the zone for the cluster.NODE_POOL_NAME
: the name of the node pool.NUMBER_NODES
: the number of nodes in the node pool in each of the cluster's zones.SURGE_NODES
: the number of extra (surge) nodes to be created on each upgrade of the node pool.UNAVAILABLE_NODES
: the number of nodes that can be unavailable at the same time on each upgrade of the node pool.To create a cluster with specific settings for surge upgrades, use the max-surge-upgrade
and max-unavailable-upgrade
flags.
gcloud container clusters create CLUSTER_NAME \ --max-surge-upgrade=SURGE_NODES --max-unavailable-upgrade=UNAVAILABLE_NODESCreating a cluster with surge upgrade disabled
To create a cluster without surge upgrades, set the value for the max-surge-upgrade
flag to 0
.
gcloud container clusters create CLUSTER_NAME \ --max-surge-upgrade=0 --max-unavailable-upgrade=1Creating a node pool with specific surge parameters
To create a node pool in an existing cluster with specific settings for surge upgrades, use the max-surge-upgrade
and max-unavailable-upgrade
flags.
gcloud container node-pools create NODE_POOL_NAME \ --num-nodes=NUMBER_NODES --cluster=CLUSTER_NAME \ --max-surge-upgrade=SURGE_NODES --max-unavailable-upgrade=UNAVAILABLE_NODESChange surge upgrade settings for an existing node pool
To update the upgrade settings of an existing node pool, use the max-surge-upgrade
and max-unavailable-upgrade
flags. If you set max-surge-upgrade
to greater than 0
, GKE creates surge nodes. If you set max-surge-upgrade
to 0
, GKE doesn't create surge nodes.
gcloud container node-pools update NODE_POOL_NAME \ --cluster=CLUSTER_NAME \ --max-surge-upgrade=SURGE_NODES --max-unavailable-upgrade=UNAVAILABLE_NODESChecking if surge upgrades are enabled on a node pool
To see if surge upgrades are enabled on a node pool, use gcloud
to describe the cluster's parameters:
gcloud container node-pools describe NODE_POOL_NAME \ --cluster=CLUSTER_NAME
If surge upgrades are enabled on the node pool, the strategy listed is SURGE
.
Note: While recreating nodes does not require additional Compute Engine resources, surge upgrading nodes does. Resource allocation is subjected to Compute Engine quota. Depending on your configuration, this quota can limit the number of parallel upgrades or even cause the upgrade to fail. If you don't have any additional quota or reservation for the nodes in your node pool, don't use maxSurge
.
For more information about quota, go to Ensure resources for node upgrades.
Configure blue-green upgradesWith blue-green node pool upgrades, you can control:
BATCH_NODE_COUNT
or BATCH_PERCENT
: the size of batches of nodes that GKE drains at a time, meaning that the Pods are removed from the nodes. Default is BATCH_NODE_COUNT=1
. If either of these settings are set to 0, GKE skips this phase and proceeds to the Soak node pool phase.BATCH_SOAK_DURATION
: the time between each batch of nodes being drained.NODE_POOL_SOAK_DURATION
: the amount of soak time for you to validate your workload on the new node configuration.To learn more about how the phases of blue-green upgrades work, see The phases of blue-green upgrades.
The following variables are used in the commands listed in the next sections:
CLUSTER_NAME
: the name of the cluster for the node pool.NODE_POOL_NAME
: the name of the node pool.NUMBER_NODES
: the number of nodes in the node pool in each of the cluster's zones.BATCH_NODE_COUNT
: the number of blue nodes to drain in a batch during the blue pool drain phase. Default is 1. If it is set to 0, the blue pool drain phase will be skipped.BATCH_PERCENT
: the percentage of blue nodes to drain in a batch during the blue pool drain phase, expressed as a decimal between 0 and 1, inclusive. GKE rounds down to the nearest node, to a minimum value of 1 node, if the percentage isn't a whole number of nodes. If it is set to 0 the blue pool drain phase will be skipped.BATCH_SOAK_DURATION
: the duration in seconds to wait after each batch drain. Default is 0.NODE_POOL_SOAK_DURATION
: the duration in seconds to wait after completing drain of all batches. Default is 3600 seconds.To create a node pool in an existing cluster with the blue-green upgrade strategy with the default parameters, use the following command:
gcloud container node-pools create NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-blue-green-upgrade
Create a node pool with blue-green upgrade using absolute node count batch sizes
To create a node pool with custom blue-green upgrade settings, use the parameter flags with the node pool creation command.
This command creates a node pool with the following customized blue-green configuration, using an absolute node count for the batch drains:
BATCH_NODE_COUNT
= 2BATCH_SOAK_DURATION
= 10sNODE_POOL_SOAK_DURATION
= 600sgcloud container node-pools create NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-blue-green-upgrade \
--standard-rollout-policy=batch-node-count=2,batch-soak-duration=10s \
--node-pool-soak-duration=600s
Create a node pool with blue-green upgrade using percentage-based batch sizes
This command creates a node pool with the following customized blue-green configuration, using a percentage for the batch drains:
BATCH_PERCENTAGE
= 25% (of the node pool size)BATCH_SOAK_DURATION
= 10sNODE_POOL_SOAK_DURATION
= 1800sgcloud container node-pools create NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-blue-green-upgrade \
--standard-rollout-policy=batch-percent=0.25,batch-soak-duration=10s \
--node-pool-soak-duration=1800s
Updating an existing node pool blue-green upgrade strategy Update a node pool with blue-green upgrade default parameters
To update an existing node pool to the blue-green upgrade strategy, use the following command:
gcloud container node-pools update NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-blue-green-upgrade
Update a node pool with blue-green upgrade using absolute node count batch sizes
To update an existing node pool to the blue-green upgrade strategy with custom settings, use the parameter flags with the node pool creation command.
This command updates a node pool to use the following customized blue-green configuration, using an absolute node count for the batch drains:
BATCH_NODE_COUNT
= 2BATCH_SOAK_DURATION
= 10sNODE_POOL_SOAK_DURATION
= 600sgcloud container node-pools update NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-blue-green-upgrade \
--standard-rollout-policy=batch-node-count=2,batch-soak-duration=10s \
--node-pool-soak-duration=600s
Update a node pool with blue-green upgrade using percentage-based batch sizes
This command creates a node pool with the following customized blue-green configuration, using a percentage for the batch drains:
BATCH_PERCENTAGE
= 25% (of the node pool size)BATCH_SOAK_DURATION
= 10sNODE_POOL_SOAK_DURATION
= 1800sgcloud container node-pools update NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-blue-green-upgrade \
--standard-rollout-policy=batch-percent=0.25,batch-soak-duration=10s \
--node-pool-soak-duration=1800s
Switching back to surge upgrades
You can change the behavior of blue-green upgrades with settings, and control the upgrade process with commands.
However, if you want to use surge upgrades instead, run the following command to switch back to surge upgrades:
gcloud container node-pools update NODE_POOL_NAME \
--cluster=CLUSTER_NAME \
--enable-surge-upgrade
Inspect the upgrade settings of a node pool
To inspect the current upgrade settings of a node pool, you can use the following command to describe the node pool:
gcloud container node-pools describe NODE_POOL_NAME \
--cluster=CLUSTER_NAME
The following snippet is an example output of the command. The strategy
field indicates the upgrade strategy in use: SURGE indicates that the surge upgrade strategy is enabled, and BLUE_GREEN indicates that the blue-green upgrade strategy is enabled.
upgradeSettings:
blueGreenSettings:
nodePoolSoakDuration: 1800s
standardRolloutPolicy:
batchNodeCount: 1
batchSoakDuration: 10s
strategy: BLUE_GREEN
This command also shows you the current phase of an in-progress blue-green upgrade. Learn more about checking the upgrade settings of a node pool.
What's nextRetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4