We are importing a .csv file with around 80.000 rows. We would like to do it on a webjob because it will be repeated in the future. We are using LinqToCsv to read in the .csv and process the information. When running locally in our Console app everything seems to be working fine. But once we deploy to the actual Azure WebJob we get an exception from LinqToCsv.
There were 1 or more exceptions while reading data using type "xxxx". Reading file "xxx.csv".
It appears it can't parse some rows in the file correctly but it does work locally. When shortening the .CSV file to 10 rows everything seems to work in production as well. So I assume nothing is wrong with the actually LinqToCSV class setup. These are the settings btw:
CsvFileDescription inputFileDescription = new CsvFileDescription
{
SeparatorChar = ';',
FirstLineHasColumnNames = true,
IgnoreUnknownColumns = true,
EnforceCsvColumnAttribute = true
};
Could someone explain this strange behaviour and how to possibility solve it without testing the .CSV row by row?
The problem was globalization. Locally my decimals were comma separated and production was a dot separation. Adding the following to the CsvFileDescription fixed my issue:
FileCultureName = "nl"