In this section we will use AWS Cost explorer to look at the costs of our EMR cluster, including the underlying EC2 Spot Instances.
Select the correct tab, depending on where you are running the workshop:
It will take 24-48 hours for your usage to appear in Cost Explorer, so you can plan to come back to this step later to check the costs of running the workshop. If your organization administrator has not granted you access to Billing information, then you will not be able to access Cost Explorer, but you can look at the examples provided below.
If you are running the workshop in an AWS event, you cannot perform this step since you do not have access to billing information in the account that was created for you. However, you can have a look at the example below, and use this step in the future as guidance on how to examine costs using Cost Explorer in your EMR workloads.
In Step 4 of the EMR cluster launch, we tagged the cluster with the following Tag: Key=Name, Value=emr-spot-workshop. This tag can be used to identify resources in your AWS account, and can also be used to identify the costs associated with usage in case the tag Key has been enabled as a Cost Allocation Tag. Click here to learn more about tagging in AWS.
AWS Cost Explorer has an easy-to-use interface that lets you visualize, understand, and manage your AWS costs and usage over time. You can analyze cost and usage data, both at a high level (e.g. how much did I pay for EMR) and for highly-specific requests (e.g. Cost for a specific instance type in a specific account with a specific tag).
If the Name tag Key was not enabled as a Cost Allocation Tag, you will not be able to filter/group according to it in Cost Explorer, but you can still gather data like cost for the EMR service, instance types, etc.
Let’s use Cost Explorer to analyze the costs of running our EMR application.
Above’s screenshot references to a sample workload deployed at an European region. You’ll get a similar graph regardless of the region.
If you have access to Cost Explorer, have a look around and see what you can find by slicing and dicing with filtering and grouping. For example, what happens if you filter by Purchase Option = Spot & Group by = Instance Type?