Search code examples
c#loopslogfilememory-efficient

How to read a log file which is hourly updated in c#?


I'm trying to write a console app in C# which reads a log file. The problem that i'm facing is that this log file is updated every 1 hour so for example, if I had 10 lines in the beginning and afterwards 12, in my second read attempt i will have to read only the 2 newly added lines. Can you suggest me a way to do this efficiently (without the need to read all the lines again because the log file usually has 5000+ lines)?


Solution

  • First of all you can use FileSystemWatcher to have notifications after file changed.

    Morover you can use FileStream and Seek function to ready only new added lines. On http://www.codeproject.com/Articles/7568/Tail-NET there is an example with Thread.Sleep:

    using ( StreamReader reader = new StreamReader(new FileStream(fileName, 
                         FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) )
    {
        //start at the end of the file
        long lastMaxOffset = reader.BaseStream.Length;
    
        while ( true )
        {
            System.Threading.Thread.Sleep(100);
    
            //if the file size has not changed, idle
            if ( reader.BaseStream.Length == lastMaxOffset )
                continue;
    
            //seek to the last max offset
            reader.BaseStream.Seek(lastMaxOffset, SeekOrigin.Begin);
    
            //read out of the file until the EOF
            string line = "";
            while ( (line = reader.ReadLine()) != null )
                Console.WriteLine(line);
    
            //update the last max offset
            lastMaxOffset = reader.BaseStream.Position;
        }
    }