AWS Kinesis Lambda Transformation using java - One or more record Ids were not returned

Question

Getting below error while processing AWS kinesis - Lambda function to S3

One or more record Ids were not returned. Ensure that the Lambda function returns all received record Ids.

The below are my code snippet.

{
        List<KinesisFirehoseOutputRecord> results = event.getRecords().stream()
                .map(record -> {
                    KinesisFirehoseOutputRecord outRec = new KinesisFirehoseOutputRecord();
                    outRec.setRecordId(record.getRecordId());
                    outRec.setData(record.getData());
                    if (record.getData().toLowerCase().contains("moldovan")) {
                        outRec.setResult("Ok");
                    } else {
                        outRec.setResult("Dropped");
                    }


                    return outRec;
                }).collect(Collectors.toList());

        return new KinesisFirehoseResponse(results);
        }

if i remove if else condition and add outRec.setResult("Ok");, it is working fine as expected. Any idea how to resolve this?

Answer 1

Firehose do a check after the function is executed... the number of recordID processed equals to the recordID that was sended to the function.

The check is doing with the array that is returned al end of the function.

If you drop any element in the process, you must be assign with the status dropped.

https://github.com/awsdocs/amazon-kinesis-data-firehose-developer-guide/blob/master/doc_source/data-transformation.md#data-transformation-status-model