Search code examples
apache-kafkaaws-lambdaterraformsasl

Lambda and Self-Managed Kafka authentication


I want to create Lambda function that will act as a consumer for self-managed Kafka messages.

I am also using Terraform, but the doc ( https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_event_source_mapping ) says: For Self Managed Kafka you will need to supply blocks for type VPC_SUBNET and VPC_SECURITY_GROUP.

What I want is to use SASL auth. From AWS it is supported on the UI.

Here is the image where you can pick sasl for self-managed kafka

Does anyone know how can I use SASL to authenticate Lambda to self-managed Kafka using Terraform?


Solution

  • From what I understand, self_managed_kafka_event_source_config should allow you to set arbitrary consumer configs for the Kafka client.

    Additional configuration block for Self Managed Kafka sources. Incompatible with "event_source_arn" and "amazon_managed_kafka_event_source_config"

    But, it only supports consumer group id, not auth. Docs: https://docs.aws.amazon.com/lambda/latest/dg/API_AmazonManagedKafkaEventSourceConfig.html