I have a bunch of .csv files in Azure blob storage, and an ingestion rule to pull them into Kusto (Azure Data Explorer). This used to work, but I've been getting a lot of ingestion failures lately. ".show ingestion failures" gives me:
Details FailureKind OperationKind ErrorCode ShouldRetry IngestionProperties IngestionSourcePath
BadRequest_EmptyArchive: Empty zip archive Permanent DataIngestPull BadRequest_EmptyArchive 0 "[Format=Csv/mandatory, IngestionMapping=[{""column"":""CrashSource"",""datatype"":""string"",""Ordinal"":""0""},{""column"":""CrashType"",""datatype"":""string"",""Ordinal"":""1""},{""column"":""ReportId"",""datatype"":""string"",""Ordinal"":""2""},{""column"":""DeviceId"",""datatype"":""string"",""Ordinal"":""3""},{""column"":""DeviceSerialNumber"",""datatype"":""string"",""Ordinal"":""4""},{""column"":""DumpFilePath"",""datatype"":""string"",""Ordinal"":""5""},{""column"":""FailureXmlPath"",""datatype"":""string"",""Ordinal"":""6""},{""column"":""PROCESS_NAME"",""datatype"":""string"",""Ordinal"":""7""},{""column"":""BUILD_VERSION_STRING"",""datatype"":""string"",""Ordinal"":""8""},{""column"":""DUMP_TYPE"",""datatype"":""string"",""Ordinal"":""9""},{""column"":""PRIMARY_PROBLEM_CLASS"",""datatype"":""string"",""Ordinal"":""10""},{""column"":""IMAGE_NAME"",""datatype"":""string"",""Ordinal"":""11""},{""column"":""FAILURE_BUCKET_ID"",""datatype"":""string"",""Ordinal"":""12""},{""column"":""OS_VERSION"",""datatype"":""string"",""Ordinal"":""13""},{""column"":""TARGET_TIME"",""datatype"":""string"",""Ordinal"":""14""},{""column"":""FAILURE_ID_HASH_STRING"",""datatype"":""string"",""Ordinal"":""15""},{""column"":""FAILURE_ID_HASH"",""datatype"":""string"",""Ordinal"":""16""},{""column"":""FAILURE_ID_REPORT_LINK"",""datatype"":""string"",""Ordinal"":""17""}], ValidationPolicy=[Options=ValidateCsvInputConstantColumns, Implications=BestEffort], Tags=[ToStringEmpty], IngestIfNotExists=[ToStringEmpty], ZipPattern=[null]]" https://crashanalysisresults.blob.core.usgovcloudapi.net/datacontainer/Telemetry.37c92f1a-a951-4047-b839-e685bd11758f.zip.crashanalysis.csv
My CSV files are not zipped in blob storage. Do I need to do something with ZipPattern to say so? Here's what this CSV contains (many strings simplified):
CrashSource,CrashType,ReportId,DeviceId,DeviceSerialNumber,DumpFilePath,FailureXmlPath,PROCESS_NAME,BUILD_VERSION_STRING,DUMP_TYPE,PRIMARY_PROBLEM_CLASS,IMAGE_NAME,FAILURE_BUCKET_ID,OS_VERSION,TARGET_TIME,FAILURE_ID_HASH_STRING,FAILURE_ID_HASH,FAILURE_ID_REPORT_LINK
"source","type","reportid","deviceid","","dumpfilepath","failurexmlpath","process","version","1","problem class","image","bucket","version","2020-07-27T22:36:44.000Z","hash string","{b27ad816-2cb5-c004-d164-516c7a32dcef}","link"
As often happens. I seem to have found my own answer just by asking. While reviewing my question, I realized the string ".zip" is in the middle of my CSV file name. (Telemetry.37c92f1a-a951-4047-b839-e685bd11758f.zip.crashanalysis.csv) That made me wonder if Kusto is treating it differently based on that. I tested by taking the exact same file, renaming it "Telemetry.37c92f1a-a951-4047-b839-e685bd11758f.crashanalysis.csv", and uploading it. This was successfully ingested.
<forehead smack> I guess I will rename my files to get past this, but this sounds like a Kusto ingestion bug to me.