Search code examples
amazon-web-serviceslambdaamazon-cloudwatchlogs

Can I stream CloudWatch log group to lambda from another account?


Can I stream CloudWatch Log Group to a lambda from another account? I've tried to configure it from AWS Console but it shows me only the lambda from the current account.

Hint: I am not interested in the Amazon Kinesis solution. I know how to do it via subscriptions as it is described here: http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CrossAccountSubscriptions.html

I am asking precisely about lambda from a different account.

P.S. I tried to add a subscription on a log group to a lambda from another account, as it is described here, but I have received the following error:

An error occurred (AccessDeniedException) when calling the PutSubscriptionFilter
operation: Cross-account lambda invocation passing is not allowed. You must use DestinationPolicies to create cross account lambda triggers.


Solution

  • Currently, no. This includes using a CloudWatch Logs Destination

    The limits of PutSubscriptionFilter are documented here in more detail: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutSubscriptionFilter.html

    CloudWatch Logs only supports the streaming of log data to Lambda functions in the same account, despite the unclear error message that seems to suggest that it might be possible.

    I suspect that the error message:

    You must use DestinationPolicies to create cross account lambda triggers.

    Is somewhat generic to any service attempting cross-account lambda invocation, and makes sense in the case of services like SNS where you can add permissions to a topic to allow it.

    Alternatively, you might be tempted to use a CloudWatch logs Destination to send to a Lambda function, however, destinations only support Kinesis.

    The closest AWS comes to documenting this is here: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDestination.html

    Which says the following about the target ARN:

    The ARN of an Amazon Kinesis stream to which to deliver matching log events.

    Despite this, on the off chance it was still possible, I attempted to follow these instructions, substituting a Lambda function for the Kinesis stream and was met with this error on the final step, which I wasn't able to resolve.

    $ aws logs put-destination --destination-name "testDestination" --target-arn "arn:aws:lambda:us-east-1:<YOUR ACCOUNT ID HERE>:function:destinationtest" --role-arn "arn:aws:iam::<YOUR ACCOUNT ID HERE>:role/CWLtoLambdaRole"
    
    An error occurred (InvalidParameterException) when calling the PutDestination operation: Could not deliver test message to specified destination. Check if the destination is valid.
    

    From all of this, I'm forced to conclude that the answer is still no for directly connecting a Lambda function to a CloudWatch Log across accounts. However, you can use a Kinesis stream in a separate account as a destination which you can then very easily subscribe a Lambda function to.