Boto3 not copying snapshot to other regions, other options?

Question

[Very new to AWS]

Hi,

I am trying to move my EBS volume snapshot copies across regions. I have been trying to use Boto3 to move the snapshots. My objective is to move the latest snapshot from us-east-2 region to us-east-1 region automatically on a daily basis.

I have used aws configure command in terminal to setup my security credentials and set region to us-east-2.

I am using pandas to acquire the most recent snapshot-id using this code:

import boto3
import pandas as pd
from pandas.io.json.normalize import nested_to_record    
import boto.ec2


    client = boto3.client('ec2')
    aws_api_response = client.describe_snapshots(OwnerIds=['self'])
    flat = nested_to_record(aws_api_response)
    df = pd.DataFrame.from_dict(flat)
    df= df['Snapshots'].apply(pd.Series)
    insert_snap = df.loc[df['StartTime'] == max(df['StartTime']),'SnapshotId']
    insert_snap = insert_snap.reset_index(drop=True)

insert_snap returns a snapshot id something like snap-1234ABCD

I am try to use this code to move the snap shot from us-east-2 to us-east-1:

client.copy_snapshot(SourceSnapshotId='%s' %insert_snap[0],
                     SourceRegion='us-east-2',
                     DestinationRegion='us-east-1',
                     Description='This is my copied snapshot.')

The snapshot is copying in the same region using the above line.

I have also tried switching regions through aws configure command in terminal, with the same issue occurring where snapshot is being copied in the same region.

There is a bug in Boto3 that is skipping the destination parameter in the copy_snapshot() code. Information found here: https://github.com/boto/boto3/issues/886

I have tried inserting this code with into the lambda manager but keep getting error "errorMessage": "Unable to import module 'lambda_function'":

region = 'us-east-2'
ec = boto3.client('ec2',region_name=region)

def lambda_handler(event, context):
    response=ec.copy_snapshot(SourceSnapshotId='snap-xxx',
                     SourceRegion=region,
                     DestinationRegion='us-east-1',
                     Description='copied from Ohio')
    print (response)

I am out of options, what I can do to automate the transfer of snapshots in aws?

Answer 1

As per CopySnapshot - Amazon Elastic Compute Cloud:

CopySnapshot sends the snapshot copy to the regional endpoint that you send the HTTP request to, such as ec2.us-east-1.amazonaws.com (in the AWS CLI, this is specified with the --region parameter or the default region in your AWS configuration file).

Therefore, you should send the copy_snapshot() command to us-east-1, with the Source Region set to us-east-2.

If you wish to move the most recent snapshot, you could run:

import boto3

SOURCE_REGION = 'us-east-2'
DESTINATION_REGION = 'us-east-1'

# Connect to EC2 in Source region
source_client = boto3.client('ec2', region_name=SOURCE_REGION)

# Get a list of all snapshots, then sort them
snapshots = source_client.describe_snapshots(OwnerIds=['self'])
snapshots_sorted = sorted([(s['SnapshotId'], s['StartTime']) for s in snapshots['Snapshots']], key=lambda k: k[1])
latest_snapshot = snapshots_sorted[-1][0]

print ('Latest Snapshot ID is ' + latest_snapshot)

# Connect to EC2 in Destination region
destination_client = boto3.client('ec2', region_name=DESTINATION_REGION)

# Copy the snapshot
response = destination_client.copy_snapshot(
    SourceSnapshotId=latest_snapshot,
    SourceRegion=SOURCE_REGION,
    Description='This is my copied snapshot'
    )

print ('Copied Snapshot ID is ' + response['SnapshotId'])