Search code examples
javafile-uploadhttp-put

Upload file at pre-signed URL using HttpPut method contains extra unnecessary part

I have pre-signed the URL. And my task is to upload a CSV file at a pre-signed URL. I have written below code

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpPut;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.HttpClients;
import java.io.IOException;
import java.io.File;
public class Test {
    /**
     * Uploading file at pre-signed URL
     *
     * @throws IOException
     */
    private void uploadFileToAWSS3(String preSignedUrl) throws IOException {
        File file = new File("/Users/vmagadum/temp/test.csv");
        HttpClient httpClient = HttpClients.custom()
                .setDefaultRequestConfig(
                        RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD).build()
                ).build();
        HttpPut put = new HttpPut(preSignedUrl);
        HttpEntity entity = MultipartEntityBuilder.create()
                .addPart("file", new FileBody(file))
                .build();
        put.setEntity(entity);
        put.setHeader("Content-Type","text/csv");

        HttpResponse response = httpClient.execute(put);

        if (response.getStatusLine().getStatusCode() == 200) {
            System.out.println("File uploaded successfully at destination.");
        } else {
            System.out.println("Error occurred while uploading file.");
        }
    }
}

With the above code, I am able to upload the file at the destination but when I download that file then I can see an unnecessary part was also added.

Structure of File I want to Upload

File I want to Upload

The file I am getting after downloading from S3

File I am getting after download

How can I ignore this extra header part that was added to the file?

Solution

  • Resolved this problem after using below code

    private void uploadFileToAWSS3() throws IOException {
        File file = new File("/Users/vmagadum/SitCopiedFile/temp/details.csv");
        HttpClient httpClient = HttpClients.custom()
                .setDefaultRequestConfig(
                        RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD).build()
                ).build();
        HttpPut put = new HttpPut(PRE_SIGNED_URL);
        HttpEntity entity = EntityBuilder.create()
                .setFile(file)
                .build();
        put.setEntity(entity);
        put.setHeader("Content-Type","text/csv");

        HttpResponse response = httpClient.execute(put);

        if (response.getStatusLine().getStatusCode() == 200) {
            System.out.println("File uploaded successfully at destination.");
        } else {
            System.out.println("Error occurred while uploading file.");
        }
    }

    Findings :

    If you add the file as multipart/form-data, then the HTTP request body will create delimiters to accommodate multiple files -

    content-Disposition: form-data; name="file1"....
<file1_data>
--BMnE0Tk<delimiter--
content-Disposition: form-data; name="file2"....
<file2_data>
--BMnE0Tk<delimiter--

    Then AWS will treat this request payload as one file and saves it. Instead, we should remove multi-part/form-data from the code and create HTTP requests with the body as file ONLY.