It appears to me that my server only allows 60 files to be downloaded per second, but I have 63 of them - all tiny, YAML files. As a result, the last 3 files don't get downloaded and throw error 503. I am using Baeldung's NIO example:
public static void downloadWithJavaNIO(String fileURL, String localFilename) throws MalformedURLException {
String credit = "github.com/eugenp/tutorials/blob/master/core-java-modules/core-java-networking-2/src/main/java/com/baeldung/download/FileDownload.java";
URL url = new URL(fileURL);
try (
ReadableByteChannel readableByteChannel = Channels.newChannel(url.openStream());
FileOutputStream fileOutputStream = new FileOutputStream(localFilename);
FileChannel fileChannel = fileOutputStream.getChannel()
) {
fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
fileOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
I was thinking of saving currentTimeMillis somewhere and checking if a second had passed by the time the 61th file is pending. But are there any other good ideas?
FileChannel.transferFrom: An invocation of this method may or may not transfer all of the requested bytes.
So I am not sure whether it works in all cases. Maybe with those small files.
I would first try a correct (non-optimized) version:
public static void downloadWithJavaNIO(String fileURL, String localFilename)
throws MalformedURLException {
URL url = new URL(fileURL);
Path targetPath = Paths.get(localFilename);
try (InputStream in = url.openStream()) {
Files.copy(in, targetPath);
} catch (IOException e) {
System.getLogger(getClass().getName()).log(Level.ERROR, fileURL, e);
}
}