in our cluster we are using 1 aws on-demand instance and a varying spot instances as nodes. For our deployment with n replicas we would like to deploy exactly 1 replica on the on-demand instance and the other n-1 replicas should be deployed on the remaining spot instances, however, multiple replicas on a single spot instance should be allowed.

Scenario: Node A (On Demand Instance) Node B (Spot Instance)

Deployment with 10 replicas

Our goal is the following pod distribution: Node A: 1 Pod Node B: 9 Pods

We already tried PodAntiAffinity, however, we run into the issue that our karpenter is just creating n nodes where n is the number of replicas. This is not ideal..

Any tips are appreciated