Connection dropped by client when serving large files for download (Java, Jersey, HTTP, GET)

Question

I have a HTTP server which serves files in download, and some of these files are pretty large (can be 7 GB or more). When these files are downloaded from some networks, the connection is dropped and we find the following error in the tomcat catalina log:

org.apache.catalina.connector.ClientAbortException: java.io.IOException: Connection reset by peer
        at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
        at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
        at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:339)
        at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:418)
        at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:406)
        at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:97)
        at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229)
        at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2147)

...

SEVERE: org.glassfish.jersey.server.ServerRuntime$Responder: An I/O error has occurred while writing a response message entity to the container output stream.
org.glassfish.jersey.server.internal.process.MappableException: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe

...

(let me know if you need more log lines)

We believe these errors are due to some firewall/proxy/NAT configuration: for some networks we have also determined a threshold over which the connection drops deterministically). We need to find a way to overcome these issues without asking the client to change their configurations (it can be done, as the same clients can download the same file via HTTP from Dropbox). Also, clients must be authenticated (i.e., have a valid session).

Here is my latest code:

@GET
@Path("download")
@Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response download(@HeaderParam("x-session-token") String sSessionId, @QueryParam("filename") String sFileName)
{            
        User oUser = MyLib.GetUserFromSession(sSessionId);

        if (oUser == null) {
            return Response.status(Status.UNAUTHORIZED).build();
        }

        File oFile = getEntryFile(sFileName);
        ResponseBuilder oResponseBuilder = null;
        if(oFile == null) {
            oResponseBuilder = Response.serverError();    
        } else {
            FileStreamingOutput oStream = new FileStreamingOutput(oFile);
            oResponseBuilder = Response.ok(oStream);
            oResponseBuilder.header("Content-Disposition", "attachment; filename="+ oFile.getName());
        }
        return oResponseBuilder.build();
    } catch (Exception e) {
        e.printStackTrace();
    }
    return null;
}

FileStreamingOutput is an extension of javax.ws.rs.core.StreamingOutput:

public class FileStreamingOutput implements StreamingOutput {

    final File m_oFile;

    public FileStreamingOutput(File oFile){
        if(null==oFile) {
            throw new NullPointerException("FileStreamingOutput.FileStreamingOutput: passed a null File");
        }
        m_oFile = oFile;
    }

    @Override
    public void write(OutputStream oOutputStream) throws IOException, WebApplicationException {
        if(null == oOutputStream) {
            throw new NullPointerException("FileStreamingOutput.write: passed a null OutputStream");
        }
        InputStream oInputStream = null;
        try {
            oInputStream = new FileInputStream(m_oFile);
            long lThreshold = 2L*1024*1024*1024;
            long lSize = m_oFile.length(); 
            if(lSize > lThreshold) {
                IOUtils.copyLarge(oInputStream, oOutputStream);
            } else {
                IOUtils.copy(oInputStream, oOutputStream);
            }
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if( oOutputStream!=null ) {
                oOutputStream.flush();
                oOutputStream.close();
            }
            if( oInputStream !=null ) {
                oInputStream.close();
            }
        }
    }
}

Before using StreamingOutput I tried building the response with an OutputStream or with a File directly. Same result.

Additional requirements: we cannot introduce other frameworks (e.g. Spring...)

Any ideas on how to overcome this limitation?

Answer 1

I solved the problem simply (!) by specifying the size of the file to be transferred in the Content-Length header, i.e., by adding the following line:

oResponseBuilder.header("Content-Length", oFile.length());