camel-jetty http proxy taking too much time for large request streams (file upload)

Question

I have created a camel-jetty http proxy bridge for my project's restful services for document download/upload. These services are invoked to get/upload documents of different types, most of the times the file size is more than 100MB.

When I invoke the upload rest service(HTTP POST) directly(not routing it via camel-jetty http proxy) to upload the document of 100MB it only takes around 2-3 mins to complete the upload and receive the response back however when I route the request via camel route it takes more than 15 mins which is kind of weird as the camel route in play is nothing but a HTTP proxy.

Following are some info:

Camel Version: 2.15.1

Camel Route definition:

    <route>
        <from uri="jetty:http://0.0.0.0:8383/sqidds/document?disableStreamCache=true&amp;matchOnUriPrefix=true&amp;enableMultipartFilter=false&amp;continuationTimeout=-1" />
        <log id="incomingMessage" message="incomingMessage - \n[id = ${id}]\n [headers = ${headers}]" />
        <to uri="jetty:http://somehost:8080/sqidds/document?bridgeEndpoint=true&amp;throwExceptionOnFailure=false&amp;httpClient.timeout=3200000" />
        <log id="outgoingMessage" message="outgoingMessage - \n[id = ${id}]\n [headers = ${headers}]" />
    </route>

Camel Project POM Excerpt:

    .
    .
    .
    <dependency>
        <groupId>org.apache.camel</groupId>
        <artifactId>camel-core</artifactId>
        <version>2.15.1.redhat-621084</version>
    </dependency>
    <dependency>
        <groupId>org.apache.camel</groupId>
        <artifactId>camel-spring</artifactId>
        <version>2.15.1.redhat-621084</version>
    </dependency>
    .
    .
    .
    <dependency>
        <groupId>org.apache.camel</groupId>
        <artifactId>camel-jetty</artifactId>
        <version>2.15.1.redhat-621084</version>
    </dependency>
    <dependency>
        <groupId>org.apache.camel</groupId>
        <artifactId>camel-http</artifactId>
        <version>2.15.1.redhat-621084</version>
        <!-- use the same version as your Camel core version -->
    </dependency>
    .
    .
    .

Rest Service(Spring MVC) Code:

@RequestMapping(method = RequestMethod.POST, consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public @ResponseBody String saveFile(@RequestPart("file") MultipartFile file) throws IOException, DocumentNotSavedException {

    String targetPath = null;
    if (file != null) {
        String repoDirectoryPath = "SOME_REPO_PATH";        
        String uniqueFileName = FileUtil.getUniqueFileName(repoDirectoryPath, file.getOriginalFilename());
        File targetFile = new File(repoDirectoryPath, uniqueFileName);
        targetPath = targetFile.getCanonicalPath();

        FileUtils.copyInputStreamToFile(file.getInputStream(), targetFile);

    } else {
        log.error("File is null");
        throw new DocumentNotSavedException("File data could not be saved");
    }

    return targetPath;
}

RestClient Code:

public String putDocument(File file,String fileName) throws RestClientException{
        ResponseEntity<String> response = null;
        try {
            File file = new File("SOME_100MB_FILE.pdf");
            byte[] fileBytes = new byte[(int)file.length()];
            LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap();
            ByteArrayResource contentsAsResource = new ByteArrayResource(fileBytes, fileName) {
                @Override
                public String getFilename() {
                    return this.getDescription();
                }
            };
            map.add("file", contentsAsResource);
            httpHeaders.setContentType(MediaType.MULTIPART_FORM_DATA);

            HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<LinkedMultiValueMap<String, Object>>(map, httpHeaders);

            response = restTemplate.exchange(serverUri, HttpMethod.POST, requestEntity, String.class);
        } catch (Exception e) {
            logger.error("Exception in put document service", e);
            throw new RestClientException("Exception in put document service :",e);
        }
        return response.getBody();
}

NOTE: For a 100MB file, the camel route log for incoming message is logged within a sec after the service is invoked, however I see the outgoing log after around 15 mins. I think there might be something wrong with the producer camel route.

Answer 1

There is an improvement in the camel 2.17.0 which can use response input stream directly in http producer.

Can you try out the latest Camel 2.17.x release (2.17.3) with your route?