Out Of Memory Error with larger file uploads

Question

I've had some issues with larger file uploads to my jetty server Trace.

I'm uploading as Multipart/form-data, and fetching the file from the request using scalatra's FileUploadSupport (as below)

class foo extends ScalatraServlet with FileUploadSupport {
  configureMultipartHandling(MultipartConfig(maxFileSize = Some(1073741824)))
  post("/upload") {
    //{1}
    ... //(VALIDATION AND USER LOGIN WITH SCENTRY)
    ... //(Transactionally posts meta info to Elasticsearch and writes video to filesystem)
  }
}

I have a battery of tests and have no issue running this with smaller files ~50MB and even 3-400MB files if running the server on localhost.

However, when i host my server on a remote machine I have some transport specific issues. And, (when debugging), I never reach a breakpoint at {1}

Researching the problem I found this Which suggests reuse of the same http connection may be causing the issue. Following their advice I added the below to my servlet and on analyzing the response headers I can confirm it "took":

before("/*") {
  response.addHeader("Connection", "close")
}

My research also showed some issues with having too many form keys, however the form in question has only 4 keys, and I don't see the issue on localhost or with smaller files on the remote machine.

This upload is occurring over https (in case it's relevant) with a CA signed certificate and is accepted by google chrome as a secure, private connection.

When setting up the Server connector, I changed the idle timeout value in case that was causing any issues

httpsConnector.setIdleTimeout(300000)

Despite these modifications, I have yet to overcome this problem and I appreciate any advice you might have.

EDIT1-SOLVED-

I believed i had already assigned 4G of heap space and therefore this was a nonesene error. Turns out the version i ran on localhost was via intellij, which has its own ideas about heap size.

when running it on the remote machine with "sbt run" I had neglected to include the fork option in my build.scala. consequently sbt ignored my jvm options (can't set options on a running jvm presumably) and i was running with a 300mb heap.

Answer 1

Your trace shows an out of memory exception. Evidently you are running out of memory when uploading the files.

Have you tried increasing your heap size. Is that the difference between your remote and local server?