I'm setting up an auto scaling group (ASG) with a mixed instances policy to use multiple spot instance types. How do I limit the number of spot instance pools used by my ASG?
Spot Instance pools are defined as follows:
A set of unused EC2 instances with the same instance type (for example,
m5.large), operating system, Availability Zone, and network platform.
I understand that in my case, a spot instance pool is basically a distinct pair of availability zone and instance type.
My CloudFormation template creates an auto scaling group of 16 instances with a mixed instances policy. It uses four instance types and all availability zones. The test region us-west-2 has four availability zones. In theory, the group should be able to use up to sixteen spot isntance pools.
The stack's SpotInstancePools parameter sets the eponymous property of the ASG. I've tried setting it to various values, but it doesn't seem to directly control the number of spot instance pools used by the ASG.
CloudFormation template:
AWSTemplateFormatVersion: '2010-09-09'
Description: Testing mixed instance policies
Parameters:
SpotInstancePools:
Type: Number
Default: 1
Description: The ASG's number of spot instance pools.
ImageId:
Type: AWS::EC2::Image::Id
Default: ami-061392db613a6357b
Description: Launch template's AMI. Defaults to Amazon Linux 2.
Resources:
AutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AvailabilityZones: !GetAZs
MaxSize: 16
MinSize: 16
MixedInstancesPolicy:
InstancesDistribution:
OnDemandAllocationStrategy: prioritized
OnDemandBaseCapacity: 0
OnDemandPercentageAboveBaseCapacity: 0
SpotAllocationStrategy: lowest-price
SpotInstancePools: !Ref SpotInstancePools
SpotMaxPrice: ''
LaunchTemplate:
LaunchTemplateSpecification:
LaunchTemplateId: !Ref LaunchTemplate
Version: !GetAtt LaunchTemplate.LatestVersionNumber
Overrides:
- InstanceType: t2.small
- InstanceType: t3.small
- InstanceType: t2.medium
- InstanceType: t3.medium
LaunchTemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
LaunchTemplateData:
ImageId: !Ref ImageId
Command to create stack mixed-instances-policy-test-1, which has a SpotInstancePools count of 1:
aws cloudformation create-stack \
--stack-name mixed-instances-policy-test-1 \
--template-body file://mixed-instances-policy.yaml \
--parameters ParameterKey=SpotInstancePools,ParameterValue=1 \
--region us-west-2 \
--profile test
Command to create stack mixed-instances-policy-5, which has a SpotInstancePools count of 5:
aws cloudformation create-stack \
--stack-name mixed-instances-policy-test-5 \
--template-body file://mixed-instances-policy.yaml \
--parameters ParameterKey=SpotInstancePools,ParameterValue=5 \
--region us-west-2 \
--profile test
Command to list the number of unique spot instance pools used (replace name of stack as appropriate):
aws ec2 describe-instances \
--filters 'Name=tag:aws:cloudformation:stack-name,Values=mixed-instances-policy-test-1' \
--query 'Reservations[].Instances[].[InstanceType, Placement.AvailabilityZone]' \
--output text \
--region us-west-2 \
--profile test |
sort |
uniq --count
After waiting for each stack to finish creating, I check the number of unique spot instance pools.
Where SpotInstancePools is set to 1, I see 3 unique pools.
5 t2.small us-west-2a
5 t3.small us-west-2b
6 t3.small us-west-2c
Where SpotInstancePools is set to 5, I see 11 unique pools.
2 t2.medium us-west-2a
1 t2.medium us-west-2b
1 t2.medium us-west-2c
2 t2.small us-west-2a
2 t2.small us-west-2b
1 t2.small us-west-2c
1 t3.medium us-west-2a
1 t3.medium us-west-2b
1 t3.medium us-west-2c
2 t3.small us-west-2b
2 t3.small us-west-2c
In each case, I expect the number of pools to equal the parameter value.
What you are seeing is the normal behavior as described in the feature release notes here: https://aws.amazon.com/blogs/aws/new-ec2-auto-scaling-groups-with-multiple-instance-types-purchase-options/
The key paragraph:
Spot Allocation Strategy – Control the amount of per-AZ diversity for the Spot Instances. A larger number adds some flexibility at times when a particular instance type is in high demand within an AZ.
The weighting impact on how instances are identified and grouped is explained here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet.html#spot-instance-weighting
You can modify/force, among other features, the InstanceType and AvailabilityZone by placing some restrictions in the Overrides section of your template as noted here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-autoscaling-autoscalinggroup-instancesdistribution.html#cfn-as-mixedinstancespolicy-spotinstancepools
So, there's nothing wrong per se, but you can add some constraints to make the pools equal to the parameters if that is preferred.