AWS Lambda with AWS MSK trigger - Role and event source must be in the same account as the cloud function

Question

I'm trying to deploy a Lambda function which is triggered by AWS MSK Kafka. I came across this error - Role and event source must be in the same account as the cloud function.

Does this mean that the AWS MSK and the Lambda must be under the same account ? Isn't there a way to workaround that?

And in case there isn't a workaround:

1. will implementing an in-code Kafka consumer work?
2. what's the recommended way to trigger it? cron expression perhaps? as I won't be aware if new messages arrived to the topic.

Thanks in advance.

Answer 1

From https://aws.amazon.com/premiumsupport/knowledge-center/lambda-cross-account-kinesis-stream/

Lambda doesn't currently support cross-account triggers from Kinesis or any stream-based sources.

To get around this, you can follow the recommended architecture from the above link. While it is for Kinesis, the overall structure should remain the same for Kakfa:

1. In the same account as the Amazon MSK Kafka (Account A), create Lambda function which is triggered by it.
2. Invoke the Lambda in another account (Account B) using the Lambda function from Account A.

The above link contains a warning that some benefits of Kinesis Data Streams are not available with this solution, you will need to evaluate whether the same applies for MSK Kafka.