Merging CSV files with different headers using CSVhelper C#

Question

When trying to merge multiple .csv files from a directory into one .csv file using CSVhelper. In the directory there are 50 .csv files, among these 50 files there are two sets of file structures, one with 7 columns and one with 6. Every file has the exact same first 5 headers however depending on the file the last two columns will change.

Example of CSVfile format 1:

Example of CSVfile format 2:

Every file in the directory will hold either of these structures with different data in the columns. The output of the new file will have data from all the columns bar Action, Code and Error Message. If i use files only with the structure of example 1 the file comes together perfectly. However, if i include files with both structures and try and use 'ErrorIPAddress' from example 2 in my new file i get the following error:

An unhandled exception of type 'CsvHelper.TypeConversion.CsvTypeConverterException' occurred in CsvHelper.dll

On this line: `IEnumerable dataRecord = reader.GetRecords().ToList();

My question is: How to use columns from one file thats not in the other? I have tried mapping it with the following:Map(m => m.ErrorIPAddress).Index(5); and i believe this is the line causing me the issue as if i comment it out the error doesn't persist however obviously i won't get the data i need into the new .csv. If i try and map by name with: Map( m => m.ErrorIPAddress ).Name( "ErrorIPAddress" ); I get the error message that ErrorIPAddress is not in the .csv file which it won't be as not all files have that column.

Output .csv format:

The final column will be generated by the ErrorIPAddress column in format 2.

Answer 1

I'm assuming you are using a single class definition with all the fields that looks something like this:

public class StudentWebAccess
{
    public int StudentID { get; set; }
    public string Gender { get; set; }
    public int Grade { get; set; }        
    public int IPAddress { get; set; } // Also ErrorIPAddress?
    public DateTime DateTime { get; set; }
    public string Action { get; set; }
    public string Code { get; set; } // Also ErrorMessage?
}

So in order to read file format 2 you are using CsvClassMap but aren't matching the properties and field names correctly. It should look something like this:

public class CsvFile2Map : CsvClassMap<StudentWebAccess>
{
    public CsvFile2Map()
    {            
        Map(m => m.IPAddress).Name("ErrorIPAddress");
        Map(m => m.Code).Name("ErrorMessage");
    }
}

If your class file uses ErrorIPAddress instead of IPAddress you have to reverse the mapping.

Map(m => m.ErrorIPAddress).Name("IPAddress");