Search code examples
elasticsearchaws-lambdaterraformamazon-cloudwatchterraform-provider-aws

Amazon CloudWatch SubscriptionFilter Elastic search Terraform support


I am trying the stream cloudwatch logs to elastic search using elastic search subscription filter. I want to automate with terraform, but didn't find if terraform supports this resource type. Please let me know if it is feasible.


Solution

  • In AWS REST API nor AWS CLI there is no such thing as a subscription to ElasticSearch. Only the following subscriptions are supported:

    • An Amazon Kinesis stream belonging to the same account as the subscription filter, for same-account delivery.

    • A logical destination that belongs to a different account, for cross-account delivery.

    • An Amazon Kinesis Firehose delivery stream that belongs to the same account as the subscription filter, for same-account delivery.

    • An AWS Lambda function that belongs to the same account as the subscription filter, for same-account delivery.

    What you see in AWS Console, is console only shortcut for that. Basically, when you create a "subscription" to ES, console will just provision a lambda function and create subscription it. The lambda will get the log events and inject into the ES domain.

    Therefor, to inject logs into ES in terraform, you have to construct such "subscription" yourself. This is done through actual subscription to a lambda function. To simplify the development, you can take the lambda function that AWS creates and use that, instead of developing your own code for injecting logs to ES.