Automated zip process sometimes skips files

Question

Part of my application exports data in a series of CSV files, adds a cover note and then packs all this data into a password protected zip file, ready for dispatch.

Here's the relevant code that does the zipping.

private bool ZipFiles(string zipfile, string filename, string password)
{
    string arguments = string.Format(@"a -tzip ""{0}"" ""{1}"" -p{2}", zipfile, filename, password);

    ProcessStartInfo startInfo = new ProcessStartInfo(_zipExePath)
    {
        UseShellExecute = false,
        RedirectStandardError = true,
        RedirectStandardInput = true,
        RedirectStandardOutput = true,
        CreateNoWindow = true,
        ErrorDialog = false,
        Arguments = arguments,
        WindowStyle = ProcessWindowStyle.Hidden
    };

    try
    {
        Process.Start(startInfo);
    }
    catch (Exception ex)
    {
        Log.Error("Zip Process Failed", ex);
        return false;
    }

    return true;
}

It's called like this:

string notePath = CreateNote(task);
string zipfile = _outputPath + "\\" task.TaskNumber + ".zip";
string filewildcard = _outputPath + "\\Data\\*.csv";

if (!ZipFiles(zipfile, filewildcard, task.Password) || !ZipFiles(zipfile, notePath, task.Password))
{
    Log.Error("Failed to created zip archive for Id " + task.TaskID);
    hasError = true;
}

Most of the time, this works without issue. However, there's an intermittent issue which causes some of the target files not to end up in the archive. Most of the time when this happens the note gets included, but none of the data. Rarely, no files at all are included in the archive.

There doesn't seem to be any unifying factor in the tasks that fail. Sometimes a task will fail to produce a correct zip and then, after being reset, it can be run again and create the zip correctly.

Whenever I've tried to use the debugger to step through this code to replicate the issue, the files zip as expected. I've done this a lot of times and in the debugger it seems to work without fault every single time.

No error is logged when the files don't all zip.

So I assumed it might be a timing issue - that effectively the code was getting to the zip portion before the disk had finished outputting the data. I added a wait in between the code that generates data, and the code I've posted

Thread.Sleep(500);

That didn't fix it. So I made it longer and longer. There's now a ten minute pause, and this issue still occurs.

I'm out of ideas on what could be causing this. Does anyone have any other thoughts?

Answer 1

I think the problem may be caused by this code:

if (!ZipFiles(zipfile, filewildcard, task.Password) || !ZipFiles(zipfile, notePath, task.Password))
{
    Log.Error("Failed to created zip archive for Id " + task.TaskID);
    hasError = true;
}

When the if statement is executed, it first calls ZipFiles which will return almost immediately with a true result, it will then call ZipFiles again (second condition of the if statement) so that you have two instances of the zip utility running at almost the same time, using the same zip filename. They might be stepping on eachother's toes.

You could try adding a call to Process.WaitForExit to make sure the first instance completes before starting the second:

try
{
    Process p = Process.Start(startInfo);
    p.WaitForExit();
}
catch (Exception ex)
{
    Log.Error("Zip Process Failed", ex);
    return false;
}

The try/catch in this code will not catch any exceptions that occur in the zip utility. It will catch any exceptions that occur while starting the executable. You may wish to check the value of p.ExitCode to see if the utility had any problems and if so, return false.