Attempted to read or write protected memory. This is often an indication that other memory is corrupt, while reading CSV file using C#

Question

I am trying to search data through a large CSV file (8MB) in size and has 18033 lines, which is a requirement for Desktop Application using Windows Forms (C#) tried in .Net version 4 and 4.6.1.

the code for the function that I'm using is the following

private List<string> SearchData(string searchText)
    {
        try
        {
            List<string> dates = new List<string>();
            using (var csv = new FileStream(Config.DataPath, FileMode.Open, FileAccess.Read, FileShare.Read))
            using (BufferedStream bs = new BufferedStream(csv))
            using (StreamReader sr = new StreamReader(bs))
            {
                var lines = string.Empty;
                while ((lines = sr.ReadLine()) != null)
                {
                    if (lines.ToLower().Contains(searchText.ToLower()))
                    {
                        dates.Add(lines.Substring(0, 10));
                    }
                }
            }
            return dates;            
        }
        catch (Exception ex)
        {

            MessageBox.Show("Unable to search the file");
            ex.Data.Add("SearchText", searchText);
            Logger.Log(ex, "SearchData");
        }
        return null;
    }

But I'm getting the following exception

System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

Get the same exception while I am using File.ReadAllLines method as well. Have tried same code in VS2017 and VS 2019 in two different PC.

Odd about the error is if put a counter in the while loop it traverse upto 500 to 600 lines otherwise it traverses through the whole file and gives me exception at the end of the process. Also does not hit the catch even if I set the AccessViolationException. The code works fine in Linqpad, thus I am not sure whether the CSV file is corrupted or not.

If anyone has any idea that would be helpful. Thanks in advance.

Answer 1

I recommend you to take a look at the CsvHelper library. Accordingly to the author,

Reading records will yield results so only one record is in memory at a time.

This could avoid the memory problem that you are having.