Skipping the errors and recording downloads in kdb

Question

I am trying to use kdb q script to download file from remote source. How can I make the download keep going if there is an error? also, how can i mark it down what its downloaded in linux when there are other files in the same directory???

Here is my code:

file:("abc.csv";"def.csv");
dbdir:"/home/terry/";
dlFunc:{
  system "download.sh abc.com user /"get /remote/path/",x /",dbdir};
dlFunc each file;

Answer 1

If you're asking how to continue downloading other files if one file fails then you can put a protected eval around your dlFunc each file, e.g.

@[dlFunc;;()]each file;

You could capture the list of failed files using something like:

badfiles:();
{@[dlFunc;x;{y;badfiles,:enlist x}x]}each file;

Then inspect the badfiles list afterwards. The ones that succeeded would be:

file except badfiles