Search code examples
c#.netusingyield-return

Reading lines of text from file with Using and Yield Return


I have the method below which uses Yield Return to read large ( >1m ) lines of text from a file.

    private static IEnumerable<string> ReadLineFromFile(TextReader fileReader)
    {
        using (fileReader)
        {
            string currentLine;
            while ((currentLine = fileReader.ReadLine()) != null)
            {
                yield return currentLine;
            }
        }
    }

I need to be able to write every 10 lines returned from this method to different files.

How do I consume this method without enumerating all the lines?

Any answer is very much appreciated.


Solution

  • I think I finally got it working :-)

            var listOfBufferedLines = ReadLineFromFile(ReadFilePath);
    
            var listOfLinesInBatch = new List<string>();
            foreach (var line in listOfBufferedLines)
            {
                listOfLinesInBatch.Add(line);
    
                if (listOfLinesInBatch.Count % 1000 == 0)
                {
                    Console.WriteLine("Writing Batch.");
                    WriteLinesToFile(listOfLinesInBatch, LoadFilePath);
                    listOfLinesInBatch.Clear();
                }
            }
    
            // writing the remaining lines
            WriteLinesToFile(listOfLinesInBatch, LoadFilePath);