Capturing standard out from tail -f "follow"

Question

I am trying to capture the output from tail in follow mode, where it outputs the text as it detects changes in the file length - particularly useful for following log files as lines are added. For some reason, my call to StandardOutput.Read() is blocking until tail.exe exits completely.

Relevant code sample:

var p = new Process() {
  StartInfo = new ProcessStartInfo("tail.exe") {
    UseShellExecute = false,
    RedirectStandardOutput = true,
    Arguments = "-f c:\\test.log"
  }
};
p.Start();

// the following thread blocks until the process exits
Task.Factory.StartNew(() => p.StandardOutput.Read());
// main thread wait until child process exits
p.WaitForExit();

I have also tried using the support for the OutputDataReceived event handler which exhibits the same blocking behavior:

p.OutputDataReceived += (proc, data) => {
  if (data != null && data.Data != null) {
    Console.WriteLine(data.Data);
  }
};
p.BeginOutputReadLine();

I do have a little bit more code around the call to StandardOutput.Read(), but this simplifies the example and still exhibits the undesirable blocking behavior. Is there something else I can do to allow my code to react to the availability of data in the StandardOutput stream prior to the child application exiting?

Is this just perhaps a quirk of how tail.exe runs? I am using version 2.0 compiled as part of the UnxUtils package.

Update: this does appear to be at least partially related to quirks in tail.exe. I grabbed the binary from the GnuWin32 project as part of the CoreUtils package and the version bumped up to 5.3.0. If I use the -f option to follow without retries, I get the dreaded "bad file descriptor" issue on STDERR (easy to ignore) and the process terminates immediately. If I use the -F option to include retries it seems to work properly after the bad file descriptor message has come by and it attempts to open the file a second time.

Is there perhaps a more recent win32 build from the coreutils git repository I could try?

Answer 1

I know it is not exatly what you are asking but as James says in the comments, you could do the equivalent functionality directly in c# to save you having to launch another process.

One way you can do it is like this:

using System;
using System.IO;
using System.Text;
using System.Threading;

public class FollowingTail : IDisposable
{
    private readonly Stream _fileStream;
    private readonly Timer _timer;

    public FollowingTail(FileInfo file,
                         Encoding encoding,
                         Action<string> fileChanged)
    {

        _fileStream = new FileStream(file.FullName,
                                     FileMode.Open,
                                     FileAccess.Read,
                                     FileShare.ReadWrite);

        _timer = new Timer(o => CheckForUpdate(encoding, fileChanged),
                           null,
                           0,
                           500);
    }

    private void CheckForUpdate(Encoding encoding,
                                Action<string> fileChanged)
    {
        // Read the tail of the file off
        var tail = new StringBuilder();
        int read;
        var b = new byte[1024];
        while ((read = _fileStream.Read(b, 0, b.Length)) > 0)
        {
            tail.Append(encoding.GetString(b, 0, read));
        }

        // If we have anything notify the fileChanged callback
        // If we do not, make sure we are at the end
        if (tail.Length > 0)
        {
            fileChanged(tail.ToString());
        }
        else
        {
            _fileStream.Seek(0, SeekOrigin.End);
        }
    }

    // Not the best implementation if IDisposable but you get the idea
    // See http://msdn.microsoft.com/en-us/library/ms244737(v=vs.80).aspx
    // for how to do it properly
    public void Dispose()
    {
        _timer.Dispose();
        _fileStream.Dispose();
    }
}

Then to call for example:

new FollowingTail(new FileInfo(@"C:\test.log"),
                  Encoding.ASCII,
                  s =>
                  {
                      // Do something with the new stuff here, e.g. print it
                      Console.Write(s);
                  });