CloudFormation: unable to recreate EC2 instance with the same ElasticIp

Question

I am trying to create and update a CloudFormation stack consisting of 1 EC2 instance with 1 EIP.

For reference, I am using the nodejs sdk.

This is the rough template (I redacted parts and omitted the Outputs):

{
  AWSTemplateFormatVersion: '2010-09-09',
  Description: '',
  Parameters: {
    ClusterStackName: {
      Description: ...,
      Type: 'String',
      Default: ...,
    },
    AmiId: {
      Description: ...,
      Type: 'String',
      Default: ...,
    },
  },
  Resources: {
    ElasticNetworkInterface: {
      Type: 'AWS::EC2::NetworkInterface',
      Properties: {
        GroupSet: [...],
        SubnetId: ...,
        Tags: [...],
      },
    },
    Instance: {
      Type: 'AWS::EC2::Instance',
      DependsOn: 'ElasticNetworkInterface',
      Properties: {
        ImageId: {
          Ref: 'AmiId',
        },
        UserData: ...,
        InstanceType: ...,
        Tenancy: 'default',
        KeyName: ...,
        PropagateTagsToVolumeOnCreation: true,
        BlockDeviceMappings: [
          {
            DeviceName: ...,
            Ebs: { VolumeSize: 'x', Encrypted: true },
          },
          {
            DeviceName: ...,
            Ebs: { VolumeSize: 'y', Encrypted: true },
          },
        ],
        NetworkInterfaces: [
          {
            NetworkInterfaceId: { Ref: 'ElasticNetworkInterface' },
            DeviceIndex: '0',
          },
        ],
        Tags: [...],
      },
    },
    ElasticIpAddress: {
      Type: 'AWS::EC2::EIP',
      Properties: {
        Domain: 'vpc',
        InstanceId: { Ref: 'Instance' },
        Tags: [...],
      },
    },
  },
}

The creation works just fine, I get my EC2 instance with the network interface and the EIP. I want to be able to recreate the machine without losing the EIP, so for example if I update the AMI and update the corresponding parameter, the update fails with the error:

Interface: [eni-aabbccdd] in use. (Service: AmazonEC2; Status Code: 400; Error Code: InvalidNetworkInterface.InUse; Request ID: xxx; Proxy: null)

For my understanding, this happens because CloudFormation needs to be able to rollback safely. Is there a better way to achieve this? Would this work if I create 2 network interfaces (one for the private and one for the public) and detach the public only?

Thanks in advance

I tried using some combination of Network Attachments and DependsOn attribute but to no avail

Answer 1

My solution here is to add a secondary network interface which will be detached and attached when recreating the instance.

The most important thing to keep in mind is that in AWS EC2 instance, the primary network interface cannot be detached in any way (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#:~:text=You%20cannot%20detach%20a%20primary,network%20interface%20per%20instance%20type.)

As a consequence, the primary network interface in the CloudFormation template can only be entirely specified using the NetworkInterfaces block inside the Instance resource. This is crucial because if you define it in any another way (say you create a NetworkInterface resource and you reference it inside the NetworkInterfaces property in the instance), CloudFormation will try to keep it by detaching while if it is fully enclosed in the instance resource, it will be deleted and recreated.

This also means that the only way to retain the private ip address when recreating an instance is to create an additional NetworkInterface that we can detach and attach and also the “official” private ip address for this machine will be the one assigned to the secondary network interface (which is eth1).

The private ip address of the primary network interface will change every time the instance is recreated, but the ip addresses of the secondary (private and public) will stay the same.

Something like this is working for me:

{
  AWSTemplateFormatVersion: '2010-09-09',
  Description: '',
  Parameters: {
    ClusterStackName: {
      Description: ...,
      Type: 'String',
      Default: ...,
    },
    AmiId: {
      Description: ...,
      Type: 'String',
      Default: ...,
    },
  },
  Resources: {
    SecondaryElasticNetworkInterface: {
      Type: 'AWS::EC2::NetworkInterface',
      Properties: {
        GroupSet: [...],
        SubnetId: ...,
        Tags: [...],
      },
    },
    SecondaryNetworkInterfaceAttachment: {
      Type: 'AWS::EC2::NetworkInterfaceAttachment',
      Properties: {
        DeviceIndex: '1',
        InstanceId: { Ref: 'Instance' },
        NetworkInterfaceId: { Ref: 'SecondaryElasticNetworkInterface' },
      },
    },
    ElasticIpAddress: {
      Type: 'AWS::EC2::EIP',
      Properties: {
        Domain: 'vpc',
        Tags: [...],
      },
    },
    ElasticIpAssociation: {
      Type: 'AWS::EC2::EIPAssociation',
      Properties: {
        AllocationId: { 'Fn::GetAtt': ['ElasticIpAddress', 'AllocationId'] },
        NetworkInterfaceId: { Ref: 'SecondaryElasticNetworkInterface' },
      },
    },
    Instance: {
      Type: 'AWS::EC2::Instance',
      DependsOn: 'ElasticNetworkInterface',
      Properties: {
        ImageId: {
          Ref: 'AmiId',
        },
        UserData: ...,
        InstanceType: ...,
        Tenancy: 'default',
        KeyName: ...,
        PropagateTagsToVolumeOnCreation: true,
        BlockDeviceMappings: [
          {
            DeviceName: ...,
            Ebs: { VolumeSize: 'x', Encrypted: true },
          },
          {
            DeviceName: ...,
            Ebs: { VolumeSize: 'y', Encrypted: true },
          },
        ],
        NetworkInterfaces: [
          {
            SubnetId: ...,
            GroupSet: [...],
            DeviceIndex: '0',
          },
        ],
        Tags: [...],
      },
    },
  },
}