Problem
InferencePoolAdvancedSpec (operator/api/v1alpha1/llmmodel_types.go) exposes only schedulerConfig. nodeSelector, affinity and tolerations exist only under spec.advanced.vllm and apply to the model pod. There is no way to place the EPP Deployment or give it imagePullSecrets.
Customer report: a two-node cluster (one GPU node, one CPU node) where the EPP image became unpullable (https://github.com/Concepts-NREC/Nebari-GitOps, see the llm-serving-pack default-image issue). One EPP replica ran on the GPU node with the image already present; the other sat in ImagePullBackOff on the CPU node. They could not pin the EPP to the node that had the image, and hand-patching the EPP Deployment was reverted by the operator within ~90s.
Proposal
Add to InferencePoolAdvancedSpec:
NodeSelector map[string]string `json:"nodeSelector,omitempty"`
Tolerations []corev1.Toleration `json:"tolerations,omitempty"`
Affinity *corev1.Affinity `json:"affinity,omitempty"`
ImagePullSecrets []corev1.LocalObjectReference `json:"imagePullSecrets,omitempty"`
Replicas *int32 `json:"replicas,omitempty"`
and thread them through buildEPPDeployment in reconcilers/inferencepool.go. Replicas is included because a single-node or two-node cluster has no reason to run two EPPs; today the count is hardcoded.
Also consider an operator-level default (defaults.epp.imagePullSecrets, defaults.epp.nodeSelector) alongside defaults.epp.image, since a private mirror is a cluster-wide decision rather than a per-model one.
Problem
InferencePoolAdvancedSpec(operator/api/v1alpha1/llmmodel_types.go) exposes onlyschedulerConfig.nodeSelector,affinityandtolerationsexist only underspec.advanced.vllmand apply to the model pod. There is no way to place the EPP Deployment or give itimagePullSecrets.Customer report: a two-node cluster (one GPU node, one CPU node) where the EPP image became unpullable (https://github.com/Concepts-NREC/Nebari-GitOps, see the llm-serving-pack default-image issue). One EPP replica ran on the GPU node with the image already present; the other sat in ImagePullBackOff on the CPU node. They could not pin the EPP to the node that had the image, and hand-patching the EPP Deployment was reverted by the operator within ~90s.
Proposal
Add to
InferencePoolAdvancedSpec:and thread them through
buildEPPDeploymentinreconcilers/inferencepool.go.Replicasis included because a single-node or two-node cluster has no reason to run two EPPs; today the count is hardcoded.Also consider an operator-level default (
defaults.epp.imagePullSecrets,defaults.epp.nodeSelector) alongsidedefaults.epp.image, since a private mirror is a cluster-wide decision rather than a per-model one.