AzCopy Ignoring or recreating Checksum

Question

I'm using the Azure tool AzCopy to export data from table storage, modify the exported data, and then import the data into another table storage table. I'm using the following command to export:

AzCopy /Source:https://MYSERVER/MYTABLE/ /SourceKey:SOURCEKEY /Dest:C:\migration /Manifest:MYTABLE

Since you cannot add a filter for the export, I'm filtering the data post-export, remove data from the JSON as necessary. I'm then using the following command to import this data to another server:

AzCopy/Source:C:\export /Dest:https://MYOTHERSERVER/MYTABLE /DestType:Table /DestKey:DESTKEY /Manifest:MYTABLE EntityOperation:InsertOrReplace

These operations work fine when I do not manipulate the JSON file. When I do, however, the contents of the file are, of course, changed and the checksum in the manifest file no longer matches. When I go to do the import, I get a "file is corrupt" message.

Here is what the manifest file looks like:

"Version":2,"PayloadFormat":"Json","Checksum":5500917691400439101,"AccountName":"SERVER","TableName":"MYTABLE","Timestamp":"2017-08-25T14:10:53.7489755Z","SplitSize":0,"TotalDataFiles":1}

How can I get AzCopy to either not validate the checksum or replace the checksum?

I've tried the following code to recreate the checksum, but when I do on the original JSON, it does not match:

var md5Hash = getFileHash(file);
var checksum = convertHash(md5Hash);

private byte[] getFileHash(string filePath)
{
    using (var md5 = MD5.Create())
    {
        using (var stream = File.OpenRead(filePath))
        {
            return md5.ComputeHash(stream);
        }
    }
}

private string convertHash(byte[] data)
{
    var algorithm = MD5.Create();
    var result = BitConverter.ToUInt64(data,0);
    return result.ToString();
}

This returns 4500814390503865422.

Answer 1

AzCopy doesn't support skipping checksum validation during table import for now. BTW, the checksum recorded in manifest file is actually CRC rather than MD5, and it's calculated by aggregating CRC of all exported files rather than the single manifest file.