|
| 1 | +include::../attributes.txt[] |
| 2 | + |
| 3 | +[.topic] |
| 4 | +[#auto-static-capacity] |
| 5 | += Static-Capacity Node Pools in EKS Auto Mode |
| 6 | +:info_titleabbrev: Static-capacity node pools |
| 7 | + |
| 8 | +Amazon EKS Auto Mode supports static-capacity node pools that maintain a fixed number of nodes regardless of pod demand. Static-capacity node pools are useful for workloads that require predictable capacity, reserved instances, or specific compliance requirements where you need to maintain a consistent infrastructure footprint. |
| 9 | + |
| 10 | +Unlike dynamic node pools that scale based on pod scheduling demands, static-capacity node pools maintain the number of nodes that you have configured. |
| 11 | + |
| 12 | +== Basic example |
| 13 | + |
| 14 | +Here's a simple static-capacity node pool that maintains 5 nodes: |
| 15 | + |
| 16 | +[source,yaml] |
| 17 | +---- |
| 18 | +apiVersion: karpenter.sh/v1 |
| 19 | +kind: NodePool |
| 20 | +metadata: |
| 21 | + name: my-static-nodepool |
| 22 | +spec: |
| 23 | + replicas: 5 # Maintain exactly 5 nodes |
| 24 | + |
| 25 | + template: |
| 26 | + spec: |
| 27 | + nodeClassRef: |
| 28 | + group: eks.amazonaws.com |
| 29 | + kind: NodeClass |
| 30 | + name: default |
| 31 | +
|
| 32 | + requirements: |
| 33 | + - key: "eks.amazonaws.com/instance-category" |
| 34 | + operator: In |
| 35 | + values: ["m", "c"] |
| 36 | + - key: "topology.kubernetes.io/zone" |
| 37 | + operator: In |
| 38 | + values: ["us-west-2a", "us-west-2b"] |
| 39 | +
|
| 40 | + limits: |
| 41 | + nodes: 8 |
| 42 | +---- |
| 43 | + |
| 44 | +== Configure a static-capacity node pool |
| 45 | + |
| 46 | +To create a static-capacity node pool, set the `replicas` field in your NodePool specification. The `replicas` field defines the exact number of nodes that the node pool will maintain. |
| 47 | + |
| 48 | +== Static-capacity node pool constraints |
| 49 | + |
| 50 | +Static-capacity node pools have several important constraints and behaviors: |
| 51 | + |
| 52 | +**Configuration constraints:** |
| 53 | + |
| 54 | +* **Cannot switch modes**: Once you set `replicas` on a node pool, you cannot remove it. The node pool cannot switch between static and dynamic modes. |
| 55 | +* **Limited resource limits**: Only the `limits.nodes` field is supported in the limits section. CPU and memory limits are not applicable. |
| 56 | +* **No weight field**: The `weight` field cannot be set on static-capacity node pools since node selection is not based on priority. |
| 57 | + |
| 58 | +**Operational behavior:** |
| 59 | + |
| 60 | +* **No consolidation**: Nodes in static-capacity pools are not considered for consolidation based on utilization. |
| 61 | +* **Scaling operations**: Scale operations bypass node disruption budgets but still respect PodDisruptionBudgets. |
| 62 | +* **Node replacement**: Nodes are still replaced for drift (such as AMI updates) and expiration based on your configuration. |
| 63 | + |
| 64 | +== Scale a static-capacity node pool |
| 65 | + |
| 66 | +You can change the number of replicas in a static-capacity node pool using the `kubectl scale` command: |
| 67 | + |
| 68 | +[source,bash] |
| 69 | +---- |
| 70 | +# Scale down to 5 nodes |
| 71 | +kubectl scale nodepool static-nodepool --replicas=5 |
| 72 | +---- |
| 73 | + |
| 74 | +When scaling down, EKS Auto Mode will terminate nodes gracefully, respecting PodDisruptionBudgets and allowing running pods to be rescheduled to remaining nodes. |
| 75 | + |
| 76 | +== Monitor static-capacity node pools |
| 77 | + |
| 78 | +Use the following commands to monitor your static-capacity node pools: |
| 79 | + |
| 80 | +[source,bash] |
| 81 | +---- |
| 82 | +# View node pool status |
| 83 | +kubectl get nodepool static-nodepool |
| 84 | +
|
| 85 | +# Get detailed information including current node count |
| 86 | +kubectl describe nodepool static-nodepool |
| 87 | +
|
| 88 | +# Check the current number of nodes |
| 89 | +kubectl get nodepool static-nodepool -o jsonpath='{.status.nodes}' |
| 90 | +---- |
| 91 | + |
| 92 | +The `status.nodes` field shows the current number of nodes managed by the node pool, which should match your desired `replicas` count under normal conditions. |
| 93 | + |
| 94 | +== Example configurations |
| 95 | + |
| 96 | +=== Basic static-capacity node pool |
| 97 | + |
| 98 | +[source,yaml] |
| 99 | +---- |
| 100 | +apiVersion: karpenter.sh/v1 |
| 101 | +kind: NodePool |
| 102 | +metadata: |
| 103 | + name: basic-static |
| 104 | +spec: |
| 105 | + replicas: 5 |
| 106 | + |
| 107 | + template: |
| 108 | + spec: |
| 109 | + nodeClassRef: |
| 110 | + group: eks.amazonaws.com |
| 111 | + kind: NodeClass |
| 112 | + name: default |
| 113 | +
|
| 114 | + requirements: |
| 115 | + - key: "eks.amazonaws.com/instance-category" |
| 116 | + operator: In |
| 117 | + values: ["m"] |
| 118 | + - key: "topology.kubernetes.io/zone" |
| 119 | + operator: In |
| 120 | + values: ["us-west-2a"] |
| 121 | +
|
| 122 | + limits: |
| 123 | + nodes: 8 # Allow scaling up to 8 during operations |
| 124 | +---- |
| 125 | + |
| 126 | +=== Static-capacity with specific instance types |
| 127 | + |
| 128 | +[source,yaml] |
| 129 | +---- |
| 130 | +apiVersion: karpenter.sh/v1 |
| 131 | +kind: NodePool |
| 132 | +metadata: |
| 133 | + name: reserved-instances |
| 134 | +spec: |
| 135 | + replicas: 20 |
| 136 | + |
| 137 | + template: |
| 138 | + metadata: |
| 139 | + labels: |
| 140 | + instance-type: reserved |
| 141 | + cost-center: production |
| 142 | + spec: |
| 143 | + nodeClassRef: |
| 144 | + group: eks.amazonaws.com |
| 145 | + kind: NodeClass |
| 146 | + name: default |
| 147 | +
|
| 148 | + requirements: |
| 149 | + - key: "node.kubernetes.io/instance-type" |
| 150 | + operator: In |
| 151 | + values: ["m5.2xlarge"] # Specific instance type |
| 152 | + - key: "karpenter.sh/capacity-type" |
| 153 | + operator: In |
| 154 | + values: ["on-demand"] |
| 155 | + - key: "topology.kubernetes.io/zone" |
| 156 | + operator: In |
| 157 | + values: ["us-west-2a", "us-west-2b", "us-west-2c"] |
| 158 | +
|
| 159 | + limits: |
| 160 | + nodes: 25 |
| 161 | +
|
| 162 | + disruption: |
| 163 | + # Conservative disruption for production workloads |
| 164 | + budgets: |
| 165 | + - nodes: 10% |
| 166 | +---- |
| 167 | + |
| 168 | +=== Multi-zone static-capacity node pool |
| 169 | + |
| 170 | +[source,yaml] |
| 171 | +---- |
| 172 | +apiVersion: karpenter.sh/v1 |
| 173 | +kind: NodePool |
| 174 | +metadata: |
| 175 | + name: multi-zone-static |
| 176 | +spec: |
| 177 | + replicas: 12 # Will be distributed across specified zones |
| 178 | + |
| 179 | + template: |
| 180 | + metadata: |
| 181 | + labels: |
| 182 | + availability: high |
| 183 | + spec: |
| 184 | + nodeClassRef: |
| 185 | + group: eks.amazonaws.com |
| 186 | + kind: NodeClass |
| 187 | + name: default |
| 188 | +
|
| 189 | + requirements: |
| 190 | + - key: "eks.amazonaws.com/instance-category" |
| 191 | + operator: In |
| 192 | + values: ["c", "m"] |
| 193 | + - key: "eks.amazonaws.com/instance-cpu" |
| 194 | + operator: In |
| 195 | + values: ["8", "16"] |
| 196 | + - key: "topology.kubernetes.io/zone" |
| 197 | + operator: In |
| 198 | + values: ["us-west-2a", "us-west-2b", "us-west-2c"] |
| 199 | + - key: "karpenter.sh/capacity-type" |
| 200 | + operator: In |
| 201 | + values: ["on-demand"] |
| 202 | +
|
| 203 | + limits: |
| 204 | + nodes: 15 |
| 205 | +
|
| 206 | + disruption: |
| 207 | + budgets: |
| 208 | + - nodes: 25% |
| 209 | +---- |
| 210 | + |
| 211 | +== Best practices |
| 212 | + |
| 213 | +**Capacity planning:** |
| 214 | + |
| 215 | +* Set `limits.nodes` higher than `replicas` to allow for temporary scaling during node replacement operations. |
| 216 | +* Consider the maximum capacity needed during node drift or AMI updates when setting limits. |
| 217 | + |
| 218 | +**Instance selection:** |
| 219 | + |
| 220 | +* Use specific instance types when you have Reserved Instances or specific hardware requirements. |
| 221 | +* Avoid overly restrictive requirements that might limit instance availability during scaling. |
| 222 | + |
| 223 | +**Disruption management:** |
| 224 | + |
| 225 | +* Configure appropriate disruption budgets to balance availability with maintenance operations. |
| 226 | +* Consider your application's tolerance for node replacement when setting budget percentages. |
| 227 | + |
| 228 | +**Monitoring:** |
| 229 | + |
| 230 | +* Regularly monitor the `status.nodes` field to ensure your desired capacity is maintained. |
| 231 | +* Set up alerts for when the actual node count deviates from the desired replicas. |
| 232 | + |
| 233 | +**Zone distribution:** |
| 234 | + |
| 235 | +* For high availability, spread static capacity across multiple Availability Zones. |
| 236 | +* When you create a static-capacity node pool that spans multiple availability zones, EKS Auto Mode distributes the nodes across the specified zones, but the distribution is not guaranteed to be even. |
| 237 | +* For predictable and even distribution across availability zones, create separate static-capacity node pools, each pinned to a specific availability zone using the `topology.kubernetes.io/zone` requirement. |
| 238 | +* If you need 12 nodes evenly distributed across three zones, create three node pools with 4 replicas each, rather than one node pool with 12 replicas across three zones. |
| 239 | + |
| 240 | +== Troubleshooting |
| 241 | + |
| 242 | +**Nodes not reaching desired replicas:** |
| 243 | + |
| 244 | +* Check if the `limits.nodes` value is sufficient |
| 245 | +* Verify that your requirements don't overly constrain instance selection |
| 246 | +* Review {aws} service quotas for the instance types and regions you're using |
| 247 | + |
| 248 | +**Node replacement taking too long:** |
| 249 | + |
| 250 | +* Adjust disruption budgets to allow more concurrent replacements |
| 251 | +* Check if PodDisruptionBudgets are preventing node termination |
| 252 | + |
| 253 | +**Unexpected node termination:** |
| 254 | + |
| 255 | +* Review the `expireAfter` and `terminationGracePeriod` settings |
| 256 | +* Check for manual node terminations or {aws} maintenance events |
0 commit comments