We now have our Jenkins Master running inside our EKS cluster, and we can reach the Jenkins dashboard via an ELB. We can create jobs which will be executed by Jenkins agents in pods within our cluster, but before we do that, let’s create a dedicated Spot based nodegroup for our Jenkins agents, which will be slightly different from our existing nodegroups.
Earlier in the workshop, in the Adding Spot Workers with eksctl step, we created nodegroups that run a diversified set of Spot Instances to run our applications.Let’s create a new eksctl nodegroup configuration file called spot_nodegroup_jenkins.yml
. The Jenkins default resource requirements (Request and Limit CPU/Memory) are 512m (~0.5 vCPU) and 512Mi (~0.5 GB RAM), and since we are not going to perform any large build jobs in this workshop, we can stick to the defaults and also choose relatively small instance types that can accommodate the Jenkins agent pods.
cat <<EoF > ~/environment/spot_nodegroup_jenkins.yml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: eksworkshop-eksctl
region: $AWS_REGION
nodeGroups:
- name: jenkins-agents-2vcpu-8gb-spot
minSize: 0
maxSize: 5
desiredCapacity: 1
instancesDistribution:
instanceTypes: ["m5.large", "m5d.large", "m4.large","t3.large","t3a.large","m5a.large","t2.large"]
onDemandBaseCapacity: 0
onDemandPercentageAboveBaseCapacity: 0
spotAllocationStrategy: capacity-optimized
labels:
lifecycle: Ec2Spot
intent: jenkins-agents
aws.amazon.com/spot: "true"
tags:
k8s.io/cluster-autoscaler/node-template/label/lifecycle: Ec2Spot
k8s.io/cluster-autoscaler/node-template/label/intent: jenkins-agents
k8s.io/cluster-autoscaler/node-template/label/aws.amazon.com/spot: "true"
iam:
withAddonPolicies:
autoScaler: true
EoF
This will create a spot_nodegroup_jenkins.yml
file that we will use to instruct eksctl to create one nodegroup (EC2 Auto Scaling group), with the labels intent: jenkins-agents
and lifecycle: Ec2Spot
. The ASG will also have a custom tag key k8s.io/cluster-autoscaler/node-template/label/intent
with the value jenkins-agents
- This is in order for Kubernetes cluster-autoscaler to respect the node selector configuration that we will apply later in the module.
Since Jenkins job oriented workloads are not fault-tolerant and an EC2 Spot interruption would cause the build job to fail, we can choose the capacity-optimized allocation strategy which will provision Spot Instances for us from the capacity pools that have the lowest chances of being interrupted. This way, we increase the chances of successfully completing our Jenkins jobs when running on Spot Instances.
eksctl create nodegroup -f spot_nodegroup_jenkins.yml
The creation of the workers will take about 3 minutes.
defualt
to jenkins-agent
. We want to be able to identify the pods that are running in our clusters by name.
intent=jenkins-agents,lifecycle=Ec2Spot
in order to instruct the Jenkins agent pods to run on the dedicated node group.
Now, when Jenkins creates new pods (=agents), these will be created with a Node Selector that instructs the kube-scheduler to only deploy the pods on nodes with the above mentioned labels, which only exist in the dedicated Jenkins nodegroup.
Move to the next step in the workshop to learn how to increase the resilience of your Jenkins jobs.