Search code examples
c#asp.net-core-3.1streamreadercsvhelper

C# .net core 3.1 read a file twice CsvHelper


I am trying to read a csv file twice with C# NET CORE 3.1. I have a limit of 100k records, if more than this I should return an error, otherwise, I should process the file. I want to get the count of the records before the actual processing (second read) so that the user gets a quick response in case the records are more than the limit. The following code doesn't work as we can read the stream only once. Is there any way in which we can read it twice so the given code works fine?

private IList<TObject> TryReadCsv(IFormFile file)
{
    int maxRecordsLimit = 100000;
    var result = new List<TObject>();
    try
    {
        using StreamReader reader = new StreamReader(file.OpenReadStream(), Encoding.UTF8);
        var config = new CsvConfiguration(CultureInfo.InvariantCulture) { Delimiter = ",", Encoding = Encoding.UTF8 };
        using var csv = new CsvReader(reader, config);
        
        // First read; light read to get the number of records without any heavy processing
        int recordsCount = 0;
        while (csv.Read())
        {
            if (recordsCount > MaxRecordsLimit)
            {
                // Return the error from here that the number of records are more than the maxRecordsLimit
            }
            recordsCount++;
        }
        
        
        // Second read for processing; heavy read
        while (csv.Read())
        {
            // Do some heavy processing which takes some more time
        }
        

        return result;
    }
    catch (CsvHelperException ex)
    {
        // catch exception
    }
}

Solution

  • As @jdweng answered I needed to set the reader position to zero as:

    reader.BaseStream.Position = 0;