I'm developing a task that requires downloading files from given URLs, one of the requirement is that it could handle network failures and exceptions.
In my implementation I had a DonwloadManager that assigns a thread for every URL, so as not to be blocking.
The problem is that I am not sure how to mock failures and exceptions through the code, I tried using Quartz as scheduler and a job that only throws exception, but as far as I know the scheduler runs on different thread, so this won't affect the downloading thread at all.
Is there a way to mock the exceptions within the downloading thread ?
Here is my code for more insights:
public class DownloadManager {
int allocatedMemory = 0;
void validateURLs(String[] urls) {
if (urls.length == 0) {
throw new IllegalArgumentException("URLs List is empty");
}
}
public ArrayList<Status> downloadURLs(String[] urls, int memorySize) throws Exception {
validateURLs(urls);
ExecutorService executor = Executors.newWorkStealingPool();
CompletionService<Status> completionService = new ExecutorCompletionService<Status>(executor);
ArrayList<Status> downloadStatus = new ArrayList<Status>();
allocatedMemory = memorySize / urls.length;
for (String url : urls) {
completionService.submit(new DownloadWorker(url, allocatedMemory));
}
for(int i=0;i<urls.length;i++){
Future<Status> URLStatus = completionService.take();
System.out.println(URLStatus.get());
downloadStatus.add(URLStatus.get());
}
executor.shutdown();
return downloadStatus;
}
And this is the download Worker:
class DownloadWorker implements Callable<Status> {
static final int SUCCESS = 1, FAILURE = 0;
private int bufferSize;
private String assignedURL;
public DownloadWorker(String assignedURL, int bufferSize) {
this.bufferSize = bufferSize;
this.assignedURL = assignedURL;
}
public Status call() throws Exception {
URLConnection openConnection = new URL(assignedURL).openConnection();
openConnection.addRequestProperty("User-Agent",
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:25.0) Gecko/20100101 Firefox/39.0");
String outputFileName = assignedURL.substring(assignedURL.lastIndexOf("/") + 1);
RandomAccessFile outputFile = new RandomAccessFile(outputFileName, "rw");
ReadableByteChannel inputChannel = Channels.newChannel(openConnection.getInputStream());
FileChannel outputChannel = outputFile.getChannel();
ByteBuffer buffer = ByteBuffer.allocate(bufferSize);
while (inputChannel.read(buffer) > 0) {
buffer.flip();
for (int i = 0; i < buffer.limit(); i++) {
outputChannel.write(buffer);
}
buffer.clear();
}
outputChannel.close();
inputChannel.close();
outputFile.close();
return new Status(assignedURL, true);
}
}
Any suggestions on how to mock/test the exception programmatically ?
Update: I was thinking, instead of scheduling exceptions, it would be easier to mock the resulting Future status, so that it returns an exception.
In your DownloadWorker
, sleep for 6 seconds and when you are getting a Future
in DownloadManager
thread, set timeout
as 5 seconds. Now you will get timeout
exception.
In this way, list out Exceptions
you want to catch in DownloadManager
thread and throw them from your DownloadWorker
.
Other way is overriding ThredPoolExecutor
after execute method as per this one:
Handling exceptions from Java ExecutorService tasks
Note that FutureTask
s swallows the exceptions (if you use submit() instead of execute() on ExecutorService
. Have a look at source code. So raise exceptions properly from DownloadWorker
.