AWS-Logo_White-Color
⁃
Running EC2 Workloads at Scale with EC2 Auto Scaling
Requirements and Notes
Architecture
Starting the workshop
...At an AWS event
...On your own
Setup with CloudFormation
Using the Cloud9 Environment
Setup AWS CLI and other tools
Clone the GitHub repo
Create an EC2 launch template
Deploy the database with Amazon RDS
Deploy the AWS Elastic Load Balancer
Create an EC2 Auto Scaling Group
Seed the database with application data
Deploy to the ASG with CodeDeploy
Using Dynamic Scaling in ASG
Stress the app with AWS Systems Manager
Resilience with Spot Instances
Finished!
Cleanup
Learn more
⁃
Launching EC2 Spot Instances
Launching EC2 Spot instances via EC2 Auto Scaling Group
Launching an EC2 Spot Instance via the RunInstances API
Launching EC2 Spot Instances via Spot Fleet
Launching EC2 Spot Instances via an EC2 Fleet
Finding Running Spot Instances
Clean Up
Final Thoughts
⁃
EC2 Spot Fleet Web App
Prerequisites
Architecture
Deploy the Application Load Balancer
Launch EC2 Spot Fleet
Spot Fleet Configuration
Testing the web app
Spot Instance interruption
Spot Interruption Testing
Wrapping Up
Appendix
⁃
ECS Deep Learning using EC2 Spot Instances
Requirements
Labs
Conventions
Workshop Cleanup
Architecture
Lab-1 Setup the workshop environment on AWS
Lab-2 Build an MXNet Docker Image
Lab-3 Deploy the MXNet Container with ECS
Lab-4 Image Classification with MXNet
Lab-5 Wrap Image Classification in an ECS Task
Finish and Clean-up
⁃
Using Amazon SageMaker Managed Spot Training
Starting the workshop
...At an AWS event
...On your own
Prerequisites
Create an Amazon SageMaker Notebook Instance
Running Notebooks
Accessing JupyterLab
Configuring Libraries for Managed Spot Training
Running Notebooks
Understanding and Reporting on Savings
Cleanup
⁃
Using Spot Instances with EKS
Introduction
Kubernetes (k8s) Basics
What is Kubernetes?
Kubernetes Nodes
K8s Objects Overview
K8s Objects Detail (1/2)
K8s Objects Detail (2/2)
Kubernetes Architecture
Architectural Overview
Control Plane
Data Plane
Kubernetes Cluster Setup
Amazon EKS
EKS Cluster Creation Workflow
What happens when you create your EKS cluster
EKS Architecture for Control plane and Worker node communication
High Level
Start the workshop...
...on your own
...at an AWS event
Create a Workspace
Install Kubernetes Tools
Update to the latest AWS CLI
Attach the IAM role to your Workspace
Update IAM settings for your Workspace
Create an SSH key
Launch using eksctl
Prerequisites
Launch EKS
Test the Cluster
Helm
Install Helm CLI
Deploy the Metric server
Install Kube-ops-view
Creating Spot nodegroups on EKS
Selecting Instance Types
Adding Spot Workers with eksctl
Deploy The Node Termination Handler
Deploying the example Microservice
Monte Carlo Pi Template
Tolerations and Affinity
Deploying the Application
Autoscaling our Applications and Clusters
Configure Cluster Autoscaler (CA)
Scale a Cluster with CA
Configure Horizontal Pod Autoscaler (HPA)
Scale the Application
(Optional) Running cost optimized and resilient Jenkins jobs
Setting up Jenkins master
Setting up Jenkins agents
Increasing resilience
Running Jenkins jobs
Auto scaling Jenkins nodes
Jenkins cleanup
Cleanup
Conclusion
What Have We Accomplished
⁃
Running Spark apps with EMR on Spot Instances
Starting the workshop
...At an AWS event
...On your own
Prerequisites and initial steps
EMR Instance Fleets
Right sizing Spark executors
Selecting instance types
Launch a cluster - Step 1
Launch a cluster - Step 2
Launch a cluster - Steps 3&4
Fleet configuration options
Examining the cluster
Spot savings summary
Tracking Spot interruptions
Automations and monitoring
Verifying the app's results
Analyzing costs
(Optional) simulating recovery
Conclusions and cleanup
⁃
CI/CD and Test Workloads with EC2 Spot Instances
Workshop Preparation
Lab 1: Reduce the cost of builds using Amazon EC2 Spot Fleet
Lab 2: Deploy testing environments using Spot & Launch Templates
Code snippet: The Test Environment CloudFormation template
Code snippet: The SpotCICDWorkshop_ManageTestEnvironment Lambda function
Lab 3: Externalise state data to add resiliency to Jenkins
Lab 4: Using containers backed by Spot instance in Auto Scaling Groups
Workshop Cleanup
⁃
Nextflow with AWS Batch
Start the workshop...
...on your own
...at an AWS event
Create a Workspace
Resize Root Volume
Create an IAM role for your Workspace
Attach the IAM role to your Workspace
Disable Temporary Credentials
Install Tools
Install Nextflow
Nextflow 101
Small Workflow Example
Workflow Example using S3
AWS Batch
Overview
Dashboard View
Local Run w/ AWS Batch
Spot Compute Environment
Create Job Queues
Create RNA-Seq Docker Image
Configure Nextflow
Local Run
AWS Batch Squared
Nextflow Docker Image
On Demand
Create Workflow Queues
Create Job Definition
Batch Squared Run
Batch Squared Debug
What you learned
Clean up the Environment
⁃
Run Monte Carlo Simulations on EC2 Spot Fleet
Set up the Environment
Explore Algorithmic Trading Concepts with Jupyter
Deploy an Automated Trading Strategy on EC2 Spot Fleet
Leverage a Fully Managed Solution using AWS Batch
Clean Up
⁃
EC2 Auto Scaling with multiple instance types and purchase options
Requirements and Notes
Architecture
Starting the workshop
...At an AWS event
...On your own
Setup with CloudFormation
Using the Cloud9 Environment
Setup AWS CLI and other tools
Clone the GitHub repo
Create an EC2 launch template
Deploy the AWS Elastic Load Balancer
Create an EC2 Auto Scaling Group
Browse to the web app
Using Scheduled Scaling in ASG
Using Dynamic Scaling in ASG
Stress the app with AWS Systems Manager
Resilience with Spot Instances
Finished!
Cleanup
Learn more
More
Amazon EC2 Spot Instances
Authors
Feedback / Questions?
Resources
English
Clear History
Github Repo
EC2 Spot Workshops
Star
Fork
© 2019 Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon EC2 Spot Workshops
>
Using Spot Instances with EKS
>
Introduction
> Amazon EKS
Amazon EKS
In this section, we’ll cover the following topics:
EKS Cluster Creation Workflow
What happens when you create your EKS cluster
EKS Architecture for Control plane and Worker node communication
High Level