Applies to: AKS on Windows Server
You can choose between two IP address assignment models for your networking architecture for AKS on Windows Server. AKS supports several deployment options for Azure Kubernetes Service (AKS):
Note
The virtual networking architecture defined here for AKS Arc might be different from the underlying physical networking architecture in a data center.
Virtual IP poolA Virtual IP (VIP) pool is set of IP addresses that are mandatory for any deployment in AKS Arc. The VIP pool is a range of reserved IP addresses used to allocate IP addresses to the Kubernetes cluster API server. It guarantees that your applications on Kubernetes services are always reachable. Keep in mind that regardless of the virtual networking model and the address assignment model you choose, you must provide a VIP pool for your AKS host deployment.
The number of IP addresses in the VIP pool depends on the number of workload clusters and Kubernetes services planned for your deployment.
Depending on your networking model, the VIP pool definition differs in the following ways:
Kubernetes nodes are deployed as specialized virtual machines in AKS Arc. AKS allocates IP addresses to these virtual machines to enable communication between Kubernetes nodes.
This networking model creates a virtual network that allocates IP addresses from a statically defined address pool to all objects in your deployment. An added benefit of using static IP networking is that long-lived deployments and application workloads are guaranteed to always be reachable.
Specify the following parameters while defining a virtual network with static IP configurations:
Important
This version of AKS does not allow any network configuration changes once the AKS host or the workload cluster is deployed. In order to change the networking settings, you must start fresh by removing the workload clusters and uninstalling AKS.
Name: The name of your virtual network.
Address prefix: The IP address prefix to use for your subnet.
Gateway: The IP address of the default gateway for the subnet.
DNS server: An array of IP addresses pointing to the DNS servers to be used for the subnet. A minimum of one and a maximum of three servers can be provided.
Kubernetes node VM pool: A continuous range of IP addresses to be used for your Kubernetes node VMs.
Virtual IP pool: A continuous range of IP addresses to be used for your Kubernetes cluster API server and Kubernetes services.
Note
The VIP pool must be part of the same subnet as the Kubernetes node VM pool.
vLAN ID: The vLAN ID for the virtual network. If it's omitted, the virtual network is not tagged.
This networking model creates a virtual network that allocates IP addresses using DHCP to all objects in the deployment.
You must specify the following parameters while defining a virtual network with static IP configurations:
Name: The name of your virtual network.
Virtual IP pool: The continuous range of IP addresses to be used for your Kubernetes cluster API server and Kubernetes services.
Note
The VIP pool addresses need to be in the same subnet as the DHCP scope, and must be excluded from the DHCP scope in order to avoid address conflicts.
vLAN ID: The vLAN ID for the virtual network. If omitted, the virtual network is not tagged.
Microsoft On-premises Cloud (MOC) is the management stack that enables the virtual machines on Windows Server-based SDDC to be managed in the cloud. MOC consists of:
cloud agent
service deployed in the cluster. This agent runs on any one node in the Windows Server cluster and is configured to fail over to another node.node agent
running on every physical node.To enable communication with MOC, you must provide the IP address CIDR to be used for the service. The -cloudserviceCIDR
is a parameter in the Set-AksHciConfig
command that's used to assign the IP address to the cloud agent service and enable high availability of the cloud agent service.
The choice of an IP address for the MOC service depends on the underlying networking model used by your cluster deployment on Windows Server.
Note
The IP address allocation for the MOC service is independent of your Kubernetes virtual network model. The IP address allocation is dependent on the underlying physical network, and the IP addresses configured for the Windows Server cluster nodes in your data center.
Windows Server cluster nodes with a DHCP-based IP address allocation mode: If your cluster nodes are assigned an IP address from a DHCP server present on the physical network, then you don't need to explicitly provide an IP address to the MOC service, as the MOC service also receives an IP address from the DHCP server.
Windows Server cluster nodes with a static IP allocation model: If your cluster nodes are assigned static IP addresses, then you must explicitly provide an IP address for the MOC cloud service. The IP address for the MOC service must be in the same subnet as the IP addresses of Windows Server cluster nodes. To explicitly assign an IP address for MOC service, use the -cloudserviceCIDR
parameter in the Set-AksHciConfig
command. Make sure you enter an IP address in the CIDR format, for example: 10.11.23.45/16
.
Both DHCP and static IP provide network connectivity on your AKS on Windows Server deployment. However, there are advantages and disadvantages to each. At a high level, the following considerations apply:
DHCP - Does not guarantee long-lived IP addresses for some resource types in an AKS deployment. - Supports expansion of reserved DHCP IP addresses if your deployment gets bigger than you initially anticipated.
Static IP - Guarantees long-lived IP addresses for all resources in an AKS deployment. - Since automatic expansion of Kubernetes node IP pool is not supported, you may not be able to create new clusters if you have exhausted the Kubernetes node IP pool.
The following table compares IP address allocation for resources between static IP and DHCP networking models:
Capability Static IP DHCP Kubernetes cluster API server Assigned statically using VIP pool. Assigned statically using VIP pool. Kubernetes nodes (on virtual machines) Assigned using Kubernetes node IP pool. Assigned dynamically. Kubernetes services Assigned statically using VIP pool. Assigned statically using VIP pool. HAProxy load balancer VM Assigned using Kubernetes node IP pool. Assigned dynamically. Microsoft On-Premises Cloud Service Depends on the physical networking configuration for Windows Server cluster nodes. Depends on the physical networking configuration for Windows Server cluster nodes. VIP pool Mandatory Mandatory Kubernetes node VM IP pool Mandatory Not supported Minimum IP address reservations for an AKS deploymentRegardless of your deployment model, the number of IP addresses reserved remains the same. This section describes the number of IP addresses you need to reserve based on your AKS Arc deployment model.
Minimum IP address reservationAt a minimum, you should reserve the following number of IP addresses for your deployment:
Cluster type Control plane node Worker node For update operations Load balancer AKS host One IP N/A Two IP N/A Workload cluster One IP per node One IP per node 5 IP One IPYou should also reserve the following number of IP addresses for your VIP pool:
Resource type Number of IP addresses Cluster API server 1 per cluster Kubernetes services 1 per service Application services 1 per service plannedAs you can see, the number of required IP addresses is variable depending on the architecture of your AKS deployment, and the number of services you run on your Kubernetes cluster. We recommend reserving a minimum of 256 IP addresses (/24 subnet) for your deployment.
Walk through an example deploymentJane is an IT administrator just starting with AKS on Windows Server. She wants to deploy two Kubernetes clusters: Kubernetes cluster A and Kubernetes cluster B on her Windows Server cluster. She also wants to run a voting application on top of her cluster. This application has three instances of the front-end UI running across the two clusters and one instance of the backend database.
Based on the previous table, she must reserve:
As previously explained, Jane requires a total of 32 IP addresses to deploy the cluster. Jane should therefore reserve a /26 subnet for her virtual network.
Split reserved IP addresses based on a static IP network modelWhile the total number of reserved IP addresses remains the same, the deployment model determines how these IP addresses are divided among IP groups. The static IP network model has two IP pools:
Working with this example, Jane must further divide these IP addresses across VIP pools and Kubernetes node IP pools:
While the total number of reserved IP addresses remain the same, the deployment model determines how these IP addresses are divided among IP group(s). As discussed in the previous section, the DHCP network model has one IP scope:
Working with the previous example:
During deployment of a target cluster, a HAProxy
-based load balancer resource is created. The load balancer is configured to distribute traffic to the pods in your service on a given port. The load balancer only works at layer 4, which indicates that the service is unaware of the actual application; i.e., it can't make any additional routing considerations.
Ingress controllers work at layer 7, and are able to use more intelligent rules to distribute application traffic. A common use of an ingress controller is to route HTTP traffic to different applications based on the inbound URL.
Next stepsThis article covers some of the networking concepts for deploying AKS nodes on Windows Server. For more information, see the following articles:
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4