Search code examples
amazon-web-servicesaws-api-gatewayamazon-kinesis-firehose

What should the mapping template look like for AWS Firehose PutRecordBatch in API Gateway?


I've successfully setup an API that has Kinesis Firehose integration with AWS API Gateway using PutRecord using these instructions (https://aws.mannem.me/?p=1152 - note: it says insecure but I still clicked through since I needed it).

I'm trying to setup an API for PutRecordBatch (essentially allows for more than one record to be written at a time) but I keep getting

{ "__type": "SerializationException" }

Based on hours of research, API gateway throws that error when incoming API call format doesn't match the mapping template noted in the Integration Request. I'm struggling to figure out how to fix my mapping template.

Here's my mapping template:

{
    "StreamName": "$input.path('DeliveryStreamName')",
    "Records": [
           #foreach($elem in $input.path('$.Records'))
              {
                "Data": "$util.base64Encode($elem.Data)",
              }#if($foreach.hasNext),#end
            #end
        ]
}

Here's the test data that I'm sending:

{
    "DeliveryStreamName": "test",
    "Records": [{
        "Data": "SampleDataStringToFirehose"
    },
    {
        "Data": "SampleDataStringToFirehose2"
    }]
}

Solution

  • So dumb but the mapping template has an error: there is an extra comma in there at the end of

    "Data": "$util.base64Encode($elem.Data)",

    that's causing the issue. Below is the correct version:

    {
        "DeliveryStreamName": "$input.path('$.DeliveryStreamName')",
        "Records": [
               #foreach($elem in $input.path('$.Records'))
                  {
                    "Data": "$util.base64Encode($elem.Data)"
                  }#if($foreach.hasNext),#end
                #end
            ]
    }