I have the method below which uses Yield Return to read large ( >1m ) lines of text from a file.
private static IEnumerable<string> ReadLineFromFile(TextReader fileReader)
{
using (fileReader)
{
string currentLine;
while ((currentLine = fileReader.ReadLine()) != null)
{
yield return currentLine;
}
}
}
I need to be able to write every 10 lines returned from this method to different files.
How do I consume this method without enumerating all the lines?
Any answer is very much appreciated.
I think I finally got it working :-)
var listOfBufferedLines = ReadLineFromFile(ReadFilePath);
var listOfLinesInBatch = new List<string>();
foreach (var line in listOfBufferedLines)
{
listOfLinesInBatch.Add(line);
if (listOfLinesInBatch.Count % 1000 == 0)
{
Console.WriteLine("Writing Batch.");
WriteLinesToFile(listOfLinesInBatch, LoadFilePath);
listOfLinesInBatch.Clear();
}
}
// writing the remaining lines
WriteLinesToFile(listOfLinesInBatch, LoadFilePath);