In this section we will deploy the instance types we selected and request nodegroups that adhere to Spot diversification best practices. For that we will use eksctl create nodegroup to generate the ClusterConfig file and save the output. The ClusterConfig file will be edited with additional configuration and use it to add the new nodes to the cluster.
Let’s first create the configuration file:
eksctl create nodegroup \ --cluster=eksworkshop-eksctl \ --region=$AWS_REGION \ --dry-run \ --managed \ --spot \ --name=dev-4vcpu-16gb-spot \ --nodes=2 \ --nodes-min=1 \ --nodes-max=5 \ --node-labels="intent=apps" \ --tags="k8s.io/cluster-autoscaler/node-template/label/intent=apps" \ --instance-types m4.xlarge,m5.xlarge,m5a.xlarge,m5ad.xlarge,m5d.xlarge,t2.xlarge,t3.xlarge,t3a.xlarge \ --asg-access \ > ~/environment/spot_nodegroup_4vcpu_16gb.yml eksctl create nodegroup \ --cluster=eksworkshop-eksctl \ --region=$AWS_REGION \ --dry-run \ --managed \ --spot \ --name=dev-8vcpu-32gb-spot \ --nodes=2 \ --nodes-min=1 \ --nodes-max=5 \ --node-labels="intent=apps" \ --tags="k8s.io/cluster-autoscaler/node-template/label/intent=apps" \ --instance-types m4.2xlarge,m5.2xlarge,m5a.2xlarge,m5ad.2xlarge,m5d.2xlarge,t2.2xlarge,t3.2xlarge,t3a.2xlarge \ --asg-access \ > ~/environment/spot_nodegroup_8vcpu_32gb.yml
This will create 2 files,
spot_nodegroups_8vcpu_32gb.yml, that we will use to instruct eksctl to create two nodegroups, both with a diversified configuration.
Let’s edit the 2 configuration files before using it to create the node groups:
If you are still struggling with the implementation, the solution files are available here
Note: Remember to change the region $AWS_REGION to your region before creating the node groups.
Create the two node groups:
eksctl create nodegroup -f spot_nodegroup_4vcpu_16gb.yml
eksctl create nodegroup -f spot_nodegroup_8vcpu_32gb.yml
The creation of each node group will take about 3 minutes.
There are a few things to note in the configuration that we just used to create these nodegroups.
If you are wondering at this stage: Where is spot bidding price ? you are missing some of the changes EC2 Spot Instances had since 2017. Since November 2017 EC2 Spot price changes infrequently based on long term supply and demand of spare capacity in each pool independently. You can still set up a maxPrice in scenarios where you want to set maximum budget. By default maxPrice is set to the On-Demand price; Regardless of what the maxPrice value, Spot Instances will still be charged at the current spot market price.
Aside from familiarizing yourself with the kubectl commands below to obtain the cluster information, you should also explore your cluster using kube-ops-view and find out the nodes that were just created.
Confirm that the new nodes joined the cluster correctly. You should see the nodes added to the cluster.
kubectl get nodes
Managed node groups automatically create a label eks.amazonaws.com/capacityType to identify which nodes are Spot Instances and which are On-Demand Instances so that we can schedule the appropriate workloads to run on Spot Instances. You can use this node label to identify the lifecycle of the nodes
kubectl get nodes \ --label-columns=eks.amazonaws.com/capacityType \ --selector=eks.amazonaws.com/capacityType=SPOT
The output of this command should return nodes running on Spot Instances. The output of the command shows the CAPACITYTYPE for the current nodes is set to SPOT.
Now we will show all nodes running on On Demand Instances. The output of the command shows the CAPACITYTYPE for the current nodes is set to ON_DEMAND.
kubectl get nodes \ --label-columns=eks.amazonaws.com/capacityType \ --selector=eks.amazonaws.com/capacityType=ON_DEMAND
Explore your cluster using kube-ops-view and find out the nodes that have just been created.