I have the following method of copying files:
public static void nioCopy(File source, File destination) {
FileInputStream fis = null;
FileOutputStream fos = null;
FileChannel input = null;
FileChannel output = null;
try {
fis = new FileInputStream(source);
fos = new FileOutputStream(destination);
input = fis.getChannel();
output = fos.getChannel();
input.transferTo(0, input.size(), output);
} catch (FileNotFoundException ex) {
Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, "Can't find either of input/output files.", ex);
} catch (IOException ex) {
Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, "Can't open either of input/output file for reading/writing", ex);
} finally {
try {
fis.close();
fos.close();
input.close();
output.close();
} catch (IOException ex) {
Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, "Error closing streams", ex);
}
}
}
And I'm using to copy file but sometimes I get a silent error or undefined behavior or I just don't know how to explain it and this is what I get:
Here is my source:
-rw-r--r-- 1 nb9 team92 3.1G 2011-10-13 16:31 6443_6#5_1_6443_6#5_2.fastq.f.fq.gz
And here is the destination:
-rw-r--r-- 1 nb9 team92 2.0G 2011-10-13 16:49 6443_6#5_1_6443_6#5_2.fastq.f.fq.gz
I get no exceptions while this process is being performed and by the looks of it everything should have been successful but then when I start unpacking the file I get:
java.io.EOFException: Unexpected end of ZLIB input stream
Clearly the destination is 1 gig OFF the original mark.
The only peculiarity is that both files are on a VERY busy lustre filesystem is it possible that this is making some funny things?
The fact that is truncated at 2Gb got me suspicious. I searched and it looks like an issue with nio. It could also be that the target filesystem allows max 2Gb files.
From Java NIO: Buffers
At present, buffer sizes are limited to 2GB (the maximum positive number that can be represented in an int. An updated planned for Java 7 will allow large buffers (with the size and indexes held as a long).
Anyway, just to be sure: