What/where is the definition of AWS Prefix?
While looking for a way to list S3 endpoint CIDR, encountered the word AWS prefix list but not sure what it exactly means and where the terminology is defined.
Prefix means a word placed in front. For S3, according to Listing Keys Hierarchically Using a Prefix and Delimiter, it should be the starting path to an object.
However, apparently it refers to a IP address range. How come prefix is used for IP ranges? What is the history or reason?
This can be used both to validate a prefix list given in a variable and to obtain the CIDR blocks (IP address ranges) for the associated AWS service.
Describes available AWS services in a prefix list format, which includes the prefix list name and prefix list ID of the service and the IP address range for the service.
SERVICE="S3"
REGION="us-west-1"
$ curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | \
jq -r --arg SERVICE "$SERVICE" --arg REGION "${REGION}" '.prefixes[] \
| select(.service==$SERVICE and .region==$REGION)'
{
"ip_prefix": "52.92.48.0/22",
"region": "us-west-1",
"service": "S3"
}
{
"ip_prefix": "54.231.232.0/21",
"region": "us-west-1",
"service": "S3"
}
{
"ip_prefix": "52.219.20.0/22",
"region": "us-west-1",
"service": "S3"
}
{
"ip_prefix": "52.219.24.0/21",
"region": "us-west-1",
"service": "S3"
}
Specify the VPC in which to create the endpoint, and the service to which you're connecting. A service is identified by a prefix list—the name and ID of a service for a Region. A prefix list ID uses the form pl-xxxxxxx and a prefix list name uses the form "com.amazonaws.region.service". Use the prefix list name (service name) to create an endpoint.
what is the meaning of Prefix ?
suppose you have a network like 10.5.10.0/24 so you will have the 10.5.10 prefix in that subnet from 1 to 255 and your network address will be 10.5.10.0
I suppose (10.0.0.0/24) means (Top 24 bit part of 32 bit IP) of a network that has 254 ip addresses from 1 to 254 (0 is network and 255 is broadcast). Prefix is top 24 bit and suffix (?) is last 8 bit. List of top N bit which identifies a network is a list of IP prefix.
If what you are looking for is prefix list id for vpc endpoint like dynamodb/s3, then it is not related to IP or CIDR. As it is mentioned in the documentation:
A prefix list ID is required for creating an outbound security group rule that allows traffic from a VPC to access an AWS service through a gateway VPC endpoint.
So if do not have prefix-list id in your security group outbout for ec2 or vpc-lambda, you will get time out when connecting to dynamodb or s3.
You can get the prefix-list by running
aws ec2 describe-prefix-lists
{
"PrefixLists": [
{
"Cidrs": [
"54.231.0.0/17",
"52.216.0.0/15"
],
"PrefixListId": "pl-63c5400k",
"PrefixListName": "com.amazonaws.us-east-1.s3"
},
{
"Cidrs": [
"52.94.0.0/22",
"52.119.224.0/20"
],
"PrefixListId": "pl-02ad2a6c",
"PrefixListName": "com.amazonaws.us-east-1.dynamodb"
}
]
}
Then you can put this PrefixListId into your security group outbound via aws web console. If you use terraform for different region, it could be something like:
resource "aws_security_group_rule" "MyService_to_DynamoDB_east" {
count = "${ lower(var.region) == "us-east-1" ? 1 : 0 }"
security_group_id = "${aws_security_group.MyService_Ext_Api.id}"
description = "DynamoDB"
type = "egress"
protocol = "tcp"
from_port = 443
to_port = 443
prefix_list_ids = ["pl-02ad2a6c"]
}
resource "aws_security_group_rule" "MyService_to_DynamoDB_west" {
count = "${ lower(var.region) == "us-west-2" ? 1 : 0 }"
security_group_id = "${aws_security_group.MyService_Ext_Api.id}"
description = "DynamoDB"
type = "egress"
protocol = "tcp"
from_port = 443
to_port = 443
prefix_list_ids = ["pl-0ca54061"]
}