Search code examples
copytransfersnapshotencryptionboto3

How can I transfer a AMI from one region to another with [non-,]Encrypted snapshots. AMI platform changes from windows to other linux


Following are are two sample codes that I tried

Code1:

import boto3
session = boto3.session.Session()
ec2_us_east = session.resource('ec2')
snap = ec2_us_east.Snapshot('snap-<id>')
snap.copy(DryRun=False,SourceRegion='us-east-1',SourceSnapshotId=snap.id,Description="testB3Copy",DestinationRegion="us-west-1")

Code2:

import boto3
ec2_us_east_client = boto3.client('ec2')
ec2_us_east_client.copy_snapshot(DryRun=False,SourceRegion='us-east-1',SourceSnapshotId=<snap-id>,Description="testB3Copy",DestinationRegion="us-west-1")

Both generates same error message as shown below:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-65-464cba179f87> in <module>() 
----> 1 ec2_us_east_client.copy_snapshot(DryRun=True,SourceRegion='us-east-1',SourceSnapshotId=snap.id,Description="testB3Copy",DestinationRegion="us-west-1")

/usr/lib/python2.7/site-packages/botocore/client.pyc in _api_call(self, **kwargs)
    239                     endpoint_prefix=service_model.endpoint_prefix,
    240                     operation_name=operation_name),
--> 241                 model=operation_model, params=request_dict
    242             )
    243

/usr/lib/python2.7/site-packages/botocore/hooks.pyc in emit(self, event_name, **kwargs)
    150         for handler in handlers_to_call:
    151             logger.debug('Event %s: calling handler %s', event_name, handler)
--> 152             response = handler(**kwargs)
    153             responses.append((handler, response))
    154         return responses

TypeError: copy_snapshot_encrypted() takes exactly 3 arguments (1 given)

I'm not sure what is going wrong here. As per the documentation copy_snapshot takes 5 arguments.

Also not sure about whether it is possible to transfer Encrypted snapshot using boto2+

When you transfer Windows AMI from one region to another (by copying snapshots and registering an AMI manually or via scripts), final AMI's platform property will be set as "Other Linux" instead of "Windows" NOTE: platform windows to other Linux change can be ignored since when the virtualization_type is hvm, it doesn't matter. VM will bootup normally since Fully virtualized VMs boot by executing mbr.


Solution

  • At the moment, Boto (v2+,3+), AWS CLI or Amazon console doesn't support AMI with encrypted volume transfer between regions.

    Also Boto (v2+,3+) doesn't transparently support transfer of AMIs with encrypted snapshots

    NOTE: There is a hard limit of 5 active snapshot transfer, which probably creates a challenge for AMI Transfer feature (Not sure). But AWSCLI (latest version as of today) supports Encrypted blockdevice transfer.

    So here is what I did. I have used AWS CLI to transfer snapshot and Python-boto to manage the transfer

    1. Shellscript
       INPUT(CommandlineArgs) takes snapshotID, source region and destination region as input
       Output snapshotID or <predeterminedstring, eg: TfrFailed>(incase transfer failed due to 5ActiveSnapshotTransferAcrossRegionLimit) 
    
    2. Python (boto)
       Input(CommandlineArgs) takes AMI_ID(s)
       Logic:
        for each ami 
           collect information about snapshots
         try
           initiates transfer of each snapshot (by calling shell script which uses AWS CLI)
         catch exception (output of shell script == <predeterminedstring>
           retry after sometime (you would be here if you had hit the hard limit of 5 active snapshots
         finally (once all snapshots are transfered)
           register a new ami (optionally transfer all tags from source AMI to target AMI)
           print SRC_AMI_ID "-->" DST_AMI_ID
        done
       Output: <SourceLocation_AMI-ID> --> <TargetLocation_AMI-ID
    

    NOTE: you might see that the target AMI looses information about "platform" since when we transfer AMIs this way we are not refering to a manifest file and I couldn't figure out a way to forcefully set platform -> "Windows". But you can ignore that, and it should work just fine if you set "Virtualization_type" as "hvm".

    NOTE: If you do this and want to use "Get password feature", you probably are out of luck since this would need platform to be set as "Windows"

    Incase of paravirtual, most likely your target AMI won't bootup unless your new ami has sufficient properties configured for PVGRUB