Search code examples
javafile-ioinputstreamoutputstreamfileoutputstream

When streaming a file from a server, the file size on disk is bigger than the total bytes read, what's going on?


I'm reading a binary file from artifactory. The file size according to artifactory is 34,952,058 bytes, the totalBytes counter that's logged after reading is finished is also 34,952,058 bytes. But the file size on disk is 39,426,048 bytes. What's going on?? I've tried BufferedOutputStream, FileOutputStream and OutputStream. It's the same result every time. What am I missing?
This is what my latest code looks like at the moment:

try {
                URL url = new URL(fw.getArtifactoryUrl());
                URLConnection connection = url.openConnection();
                in = connection.getInputStream();
                File folder = utils.getFirmwareFolder(null, FirmwareUtils.FIRMWARE_LATEST, true);
                StringBuilder builder = new StringBuilder(folder.toString());
                builder.append("/").append(fw.getFileName());
                Path filePath = Paths.get(builder.toString());
                OutputStream out = Files.newOutputStream(filePath);

                int read = 0;
                int totalBytes = 0;

                while ((read = in.read(bytes)) > 0) {
                    totalBytes += read;
                    out.write(bytes);
                    out.flush();
                }
                logger.info("Total bytes read: " + totalBytes);
                in.close();
                out.close();

<<< more code >>>

Solution

  • Your code reads correctly, but writes incorrectly

    while ((read = in.read(bytes)) > 0) { // Read amount of bytes
        totalBytes += read;  // Add the correct amount of bytes read to total
        out.write(bytes);    // Write the whole array, no matter how much we read
        out.flush();         // Completely unnecessary, can harm performance
    }
    

    You need out.write(bytes, 0, read) to write only the bytes you've read instead of the whole buffer.