IAM has no access to Elasticsearch domain while Setting Opensearch as a Firehose destination

Question

I'm trying to build a firehose that delivers data to Opensearch using CDK with Python, when I do that from Console it works like a charm however when I try to deploy it with CDK it gives me this error

Resource handler returned message: "Verify that the IAM role has access to the ElasticSearch domain.

It is deriving me crazy, I tried every IAM es policy but no one works. this is my configuration here I define the role

        self.firehose_role = Role(
            self,
            "FirehoseRole",
            role_name=f"{construct_name}",
            assumed_by=ServicePrincipal(service="firehose.amazonaws.com"),
        )

        self.firehose_role.add_to_policy(
            PolicyStatement(
                actions=["es:*"],
                resources=[
                    "arn:aws:es:eu-west-1:xxx:domain/my-domain",
                    "arn:aws:es:eu-west-1:xxx:domain/my-domain/*",
                ],
            ),
        )

        self.firehose_role.add_to_policy(
            PolicyStatement(
                actions=[
                    "ec2:DescribeVpcs",
                    "ec2:DescribeVpcAttribute",
                    "ec2:DescribeSubnets",
                    "ec2:DescribeSecurityGroups",
                    "ec2:DescribeNetworkInterfaces",
                    "ec2:CreateNetworkInterface",
                    "ec2:CreateNetworkInterfacePermission",
                    "ec2:DeleteNetworkInterface",
                ],
                resources=[
                    "*",
                ],
            ),
        )

Then setting OS configuration as following

        os_config =CfnDeliveryStream.AmazonopensearchserviceDestinationConfigurationProperty(
            index_name="xxx",
            role_arn=self.firehose_role.role_arn,
            s3_configuration=CfnDeliveryStream.S3DestinationConfigurationProperty(
                #bucket config
            ),
            buffering_hints=CfnDeliveryStream.ElasticsearchBufferingHintsProperty(
                interval_in_seconds=120,
                size_in_m_bs=5,
            ),
            cloud_watch_logging_options=CfnDeliveryStream.CloudWatchLoggingOptionsProperty(
                enabled=True,
                log_group_name=log_group.log_group_name,
                log_stream_name=log_stream.log_stream_name,
            ),
            domain_arn="arn:aws:es:eu-west-1:xxx:domain/my-domain",
            retry_options=CfnDeliveryStream.AmazonopensearchserviceRetryOptionsProperty(
                duration_in_seconds=180
            ),
            s3_backup_mode="AllDocuments",
            vpc_configuration=CfnDeliveryStream.VpcConfigurationProperty(
                role_arn=self.firehose_role.role_arn,
                security_group_ids=["xxx"],
                subnet_ids=["xxx", "xxx"],
            ),
        )

To finally setting Kinesis Delivery stream

        self.delivery_stream = CfnDeliveryStream(
            self,
            id="FirehoseDS",
            delivery_stream_name=f"{construct_name}-ds",
            delivery_stream_type="DirectPut",
            amazonopensearchservice_destination_configuration=os_config,
        )

I run out of ideas, PS: knowing that OS access policy is wide open and I don't think the problem is there because if it was there at least I would've been able to deploy Firehose the I would have problems with data not reaching, but now I'm not able to even deploy Firehose

Answer 1

Apparently I managed to solve it by deploying IAM actions separately as an IAM Managed Policy, then adding the policy to Firehose role. I'm not sure why but I think CDK was trying to create Firehose before the IAM is ready to use which throws an error and prevent it from being deployed.