Use the **min** of the "karpenter-calculated" and "user-inputted" maxPods value as the final kubelet config maxPods value

[dlemfh]

What problem are you trying to solve?

Description of the original problem posted here: #3903:

Setting spec.kubeletConfiguration.maxPods to 110 caused nodes to think they could handle 110 pods even if the node could not get as many nics/ips. If the ec2 instance can support 58 pods (awslabs/amazon-eks-ami@master/files/eni-max-pods.txt) and the maxPods config is set to 110 I would expect the node to never try and run more than 58 pods.

In other words, we want to be able to rely on Karpenter's automatically calculated maxPods for small instances, but also we want to be able to put an upper bound (e.g. 110) to the maxPods value.

In order to achieve this, Karpenter behavior could be changed this way:

1. first take the "karpenter-calculated" max pod count (=58)
2. and take the "user-inputted" max pod count (=110)
3. finally, set the min of the two as the final maxPods kubelet config value

But then, some users may want to use the user-inputted max pods value to override final the decision (as is the current behavior), because of some flaw in karpenter's automated calculations.

Therefore, I think karpenter should have a flag that allows the user to decide between:

• using the "user-inputted" max pod count as the final decision to override the final kubelet config value (= the current ASIS behavior)
• or using the minimum of "user-inputted" and "karpenter-calculated" max pod counts as the final kubelet config value (= which is what will solve the original Issue Author's and my use case)

Originally posted by @dlemfh in #3903 (comment)

How important is this feature to you?

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment