Search code examples
amazon-web-servicesaws-lambdaaws-cloudformationserverlessamazon-cloudwatchlogs

"An error occurred: LogGroup - <resource name> already exists" while trying to deploy Serverless


after running sls deploy -v && sls s3deploy as I normally do, I ran into this issue:

...

CloudFormation - CREATE_FAILED - AWS::Logs::LogGroup - CallTextractLogGroup

...

CloudFormation - CREATE_FAILED - AWS::IAM::Role - IamRoleStateMachineExecution
CloudFormation - CREATE_FAILED - AWS::Logs::LogGroup - StartTextractStateMachineLogGroup
CloudFormation - CREATE_FAILED - AWS::Lambda::LayerVersion - Boto3LayerLambdaLayer

...

  Serverless Error ---------------------------------------

  An error occurred: StartTextractStateMachineLogGroup - /aws/lambda/textract-service-dev-startTextractStateMachine already exists.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     OS:                     linux
     Node Version:           12.2.0
     Serverless Version:     1.42.3

I don't remember changing anything at all before deployment, other than adding Retry sections to my Step Function States, which shouldn't have messed with CF logs at all.

I tried running sls remove and then my deployment command again, no luck there.

Also tried adding cfLogs: true to my provider section in the YAML, no love!

I tried manually deleting the LogGroup in the CloudWatch console, but it's not even there.

Here is my serverless.yml:

service: textract-service

provider:
  name: aws
  runtime: python3.7
  timeout: 10
  region: us-east-1
  cfLogs: true
  environment:
    STATE_MACHINE_ARN: ${self:resources.Outputs.TextractStepFunctions.Value}
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:PutObject"
        - "states:*"
      Resource:
        Fn::Join:
          - ""
          - - "the-chumiest-bucket-ARN or *"
            - ${self:resources.Outputs.TextractStepFunctions.Value}

plugins:
  - serverless-plugin-existing-s3
  - serverless-step-functions
  - serverless-pseudo-parameters
  - serverless-plugin-existing-s3 
layers:
  boto3Layer:
    package:
      artifact: boto3_layer.zip
    allowedAccounts:
      - "*"

functions:
  startTextractStateMachine:
    handler: src/start_textract_state_machine.lambda_handler
    role: the-chumiest-bucket-role
    layers:
      - {Ref: Boto3LayerLambdaLayer}
    events:
      - existingS3:
          bucket: the-chumiest-bucket
          events:
            - s3:ObjectCreated:*
          rules:
            - prefix: input1/
            - suffix: .pdf
  callTextract:
    handler: src/call_textract.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}
  getTextractOutput:
    handler: src/get_textract_output.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}
  parseTextractOutput:
    handler: src/parse_textract_output.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}

stepFunctions:
  stateMachines:
    textractStepFunc:
      name: TextractStepFunctions
      definition:
        Comment: A state machine for the Textract OCR process.
        StartAt: StartTextractStateMachine
        States:
          StartTextractStateMachine:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-startTextractStateMachine"
            Next: CallTextract
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          CallTextract:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-callTextract"
            Next: GetTextractOutput
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          GetTextractOutput:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-getTextractOutput"
            Next: ParseTextractOutput
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          ParseTextractOutput:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-parseTextractOutput"
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
            End: true

resources:
  Outputs:
    TextractStepFunctions:
      Description: The ARN of the state machine
      Value:
        Ref: TextractStepFunctions

Solution

  • It looks like you have a log group from a previous (failed?) deployment that still exists in CloudWatch Logs.

    You should see this log group in the CloudWatch console (not CloudFormation).

    You can run this command (AWS CLI):

    aws logs delete-log-group --log-group-name /aws/lambda/textract-service-dev-startTextractStateMachine
    

    to delete your log group and then retry to deploy.