groovy aws-api-gateway multipartform-data

Parsing an APIGatewayProxyRequestEvent in Groovy, that contains multipart/form-data

I am trying to convert my command-line app, which cleans up CSV/Excel files, into an AWS Lambda Function.

I was able to get it uploaded to AWS Lambda, but I faced some issues over some dependencies...

The test, and code, in question

To test against this, I decided to write a unit-test (in Spock) against the offending part of the Function code...

The test itself

package com.mikebuyshouses.dncscrubber.awsServices

import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent
import com.mikebuyshouses.dncscrubber.awsServices.providers.TestPathProvider
import com.mikebuyshouses.dncscrubber.utils.FileUtils
import spock.lang.Specification

class ApiRequestHandlerTest extends Specification {
    def "parseRequestEvent should actually write the input file from the request event"() {
        setup:
        def stubRequestEvent = Stub(APIGatewayProxyRequestEvent)
        stubRequestEvent.getHeaders() >> ["Content-Type": "multipart/form-data;boundary=YRPyKRi1Bb3sIDcW"]
        stubRequestEvent.getIsBase64Encoded() >> true;
        stubRequestEvent.getBody() >> """
------WebKitFormBoundaryYRPyKRi1Bb3sIDcW\r
Content-Disposition: form-data; name="inputFile"; filename="test.csv"\r
Content-Type: text/csv\r
\r
Number,Sale Date,Address,City,State,Zip,Parcel,Township,Removed,Removed Date,Bankruptcy #,Redemption Days,Cause #,Receive Date,Receive Time,Total Judgement,Plaintiff,Et al,Attorney,Phone,User Fee,Sheriff Fee,Ad Cost,Delinquent Tax,Total Fees,Defendant,Judgement,Interest 1,Interest 2,Additional Charges,Lien Holder 1,Lien Holder 1 Judgement,Lien Holder 1 Interest 1,Lien Holder 1 Interest 2,Lien Holder 1 Additional Charges,Lien Holder 2,Lien Holder 2 Judgement,Lien Holder 2 Interest 1,Lien Holder 2 Interest 2,Lien Holder 2 Additional Charges,Lien Holder 3,Lien Holder 3 Judgement,Lien Holder 3 Interest 1,Lien Holder 3 Interest 2,Lien Holder 3 Additional Charges,Lien Holder 4,Lien Holder 4 Judgement,Lien Holder 4 Interest 1,Lien Holder 4 Interest 2,Lien Holder 4 Additional Charges,Lien Holder 5,Lien Holder 5 Judgement,Lien Holder 5 Interest 1,Lien Holder 5 Interest 2,Lien Holder 5 Additional Charges,Lien Holder 6,Lien Holder 6 Judgement,Lien Holder 6 Interest 1,Lien Holder 6 Interest 2,Lien Holder 6 Additional Charges,Lien Holder 7,Lien Holder 7 Judgement,Lien Holder 7 Interest 1,Lien Holder 7 Interest 2,Lien Holder 7 Additional Charges,Lien Holder 8,Lien Holder 8 Judgement,Lien Holder 8 Interest 1,Lien Holder 8 Interest 2,Lien Holder 8 Additional Charges,Lien Holder 9,Lien Holder 9 Judgement,Lien Holder 9 Interest 1,Lien Holder 9 Interest 2,Lien Holder 9 Additional Charges,,Grand Total\r
1,09/15/2023,1001 Main Street,Indianapolis,IN,46220,,BRO,N,,,,123456789MF123456,07/14/2023,,44202.67,FAKE BANK,,CARLISLE,(216) 555-1234,300,66,345.42,0,711.42,DOUGLAS H LOPEZ,44202.67,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,44914.09\r
\r
------WebKitFormBoundaryYRPyKRi1Bb3sIDcW--\r
""".bytes.encodeBase64()

        ApiRequestHandler apiRequestHandler = new ApiRequestHandler().with { ApiRequestHandler handler ->
            handler.pathProvider = new TestPathProvider();

            return handler;
        }

        when:
        apiRequestHandler.parseRequestEvent(stubRequestEvent)

        then:
        new File("${FileUtils.TestDirectoryPath}/${FileUtils.InputPathPart}/test.csv").exists()

        cleanup:
        new File("${FileUtils.TestDirectoryPath}/${FileUtils.InputPathPart}/test.csv").delete()
    }

}

The code under test

This code tests the following method-in-question:

RequestBodyModel parseRequestEvent(APIGatewayProxyRequestEvent request) {
    byte[] bodyBytes = request.getBody().getBytes("UTF-8");
    if (request.getIsBase64Encoded())
        bodyBytes = Base64.getDecoder().decode(bodyBytes);

    RequestBodyModel model = new RequestBodyModel();

    FileUpload.parse(bodyBytes, request.getHeaders().get('Content-Type'))
        .each { FileItem fileItem ->
            if (fileItem.getFieldName().equals("inputFile")) {
                File inputFile = FileUtils.CreateFileIfNotExists("${this.pathProvider.getBaseInputPath()}/${fileItem.getName()}");
                fileItem.write(inputFile);
                model.inputFile = inputFile;
                return;
            }

            if (fileItem.getFieldName().equals("outputFileExtension")) {
                model.outputFileExtension = fileItem.getString();
                return;
            }

            if (fileItem.getFieldName().equals("shouldExportDncRecords")) {
                model.shouldExportDncRecords = Boolean.valueOf(fileItem.getString());
                return;
            }
        }

    return model;
}

The model being built in question, RequestBodyModel is defined to be:

class RequestBodyModel {
    File inputFile;
    boolean shouldExportDncRecords;
    String outputFileExtension;
}

What happens when you run the test

It fails. The file can't be found, and when I debug the project, including the FileUpload.parse(), I see no FileItems returned!

Libraries used

delight-fileupload, which uses...
.... Apache Commons FileUpload
AWS Lambda Java Events
Spock (for unit testing)

Burning Question

How can I parse my multipart/form-data request, defined in the test above, using the FileUpload (or something similar)? I want to work with FileItems...

Solution

Found the answer:

It was my boundary that was incorrect.

I adjust the stub request logic to the following:

        setup:
        String multipartBoundary = "----WebKitFormBoundaryYRPyKRi1Bb3sIDcW";
        def stubRequestEvent = Stub(APIGatewayProxyRequestEvent)
        stubRequestEvent.getHeaders() >> ["Content-Type": "multipart/form-data;boundary=${multipartBoundary}"]
        stubRequestEvent.getIsBase64Encoded() >> true;
        stubRequestEvent.getBody() >> """
--${multipartBoundary}\r
Content-Disposition: form-data; name="inputFile"; filename="test.csv"\r
Content-Type: text/csv\r
\r
Number,Sale Date,Address,City,State,Zip,Parcel,Township,Removed,Removed Date,Bankruptcy #,Redemption Days,Cause #,Receive Date,Receive Time,Total Judgement,Plaintiff,Et al,Attorney,Phone,User Fee,Sheriff Fee,Ad Cost,Delinquent Tax,Total Fees,Defendant,Judgement,Interest 1,Interest 2,Additional Charges,Lien Holder 1,Lien Holder 1 Judgement,Lien Holder 1 Interest 1,Lien Holder 1 Interest 2,Lien Holder 1 Additional Charges,Lien Holder 2,Lien Holder 2 Judgement,Lien Holder 2 Interest 1,Lien Holder 2 Interest 2,Lien Holder 2 Additional Charges,Lien Holder 3,Lien Holder 3 Judgement,Lien Holder 3 Interest 1,Lien Holder 3 Interest 2,Lien Holder 3 Additional Charges,Lien Holder 4,Lien Holder 4 Judgement,Lien Holder 4 Interest 1,Lien Holder 4 Interest 2,Lien Holder 4 Additional Charges,Lien Holder 5,Lien Holder 5 Judgement,Lien Holder 5 Interest 1,Lien Holder 5 Interest 2,Lien Holder 5 Additional Charges,Lien Holder 6,Lien Holder 6 Judgement,Lien Holder 6 Interest 1,Lien Holder 6 Interest 2,Lien Holder 6 Additional Charges,Lien Holder 7,Lien Holder 7 Judgement,Lien Holder 7 Interest 1,Lien Holder 7 Interest 2,Lien Holder 7 Additional Charges,Lien Holder 8,Lien Holder 8 Judgement,Lien Holder 8 Interest 1,Lien Holder 8 Interest 2,Lien Holder 8 Additional Charges,Lien Holder 9,Lien Holder 9 Judgement,Lien Holder 9 Interest 1,Lien Holder 9 Interest 2,Lien Holder 9 Additional Charges,,Grand Total\r
1,09/15/2023,1001 Main Street,Indianapolis,IN,46220,,BRO,N,,,,123456789MF123456,07/14/2023,,44202.67,FAKE BANK,,CARLISLE,(216) 555-1234,300,66,345.42,0,711.42,DOUGLAS H LOPEZ,44202.67,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,0,0,0,0,,44914.09\r
\r
--${multipartBoundary}--\r
""".bytes.encodeBase64()

re-ran the test case, and it passed!