My solution to responding to syslog events stops working after several hours. I was running 'tail -f' from C# and subscribing to DataReceived events. My issue is that after several hours it seems to just stop working without throwing errors or exiting the process. I think this happens when the syslog file rolls over.
Current solution that stops working after many hours:
using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
using App.Configuration;
using App.LogData;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
namespace IQIngest.SrtWatch.Services
{
public class SyslogReaderService
{
private ILogger<SyslogReaderService> _logger;
private CancellationTokenSource _cancellationTokenSource;
private IOptionsSnapshot<AppSettings> _appSettings;
private ITargetBlock<string> _targetQueue;
public SyslogReaderService(ILogger<SyslogReaderService> logger,
CancellationTokenSource cancellationTokenSource,
IOptionsSnapshot<AppSettings> appSettings,
ITargetBlock<string> targetQueue)
{
_logger = logger;
_cancellationTokenSource = cancellationTokenSource;
_appSettings = appSettings;
_targetQueue = targetQueue;
}
public async Task StartReader()
{
ProcessStartInfo startInfo = new ProcessStartInfo($"/usr/bin/tail", $"-f {_appSettings.Value.TailedFile}")
{
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true
};
await Task.Factory.StartNew(() =>
{
try
{
using (Process tail = new Process()
{
StartInfo = startInfo
})
{
_cancellationTokenSource.Token.Register(tail.Kill);
tail.OutputDataReceived += handleLine;
tail.ErrorDataReceived += handleLine;
tail.Start();
_logger.LogDebug($"Started tail process for file {{@{LogFields.FILENAME_STR}}}.", _appSettings.Value.TailedFile);
tail.BeginOutputReadLine();
tail.BeginErrorReadLine();
tail.WaitForExit();
//Stops working and doesn't make it to this line.
_logger.LogDebug($"Tail stopped.");
if (!_cancellationTokenSource.IsCancellationRequested) _cancellationTokenSource.Cancel();
}
}
catch (Exception ex)
{
_logger.LogError(ex, $"Unhandled exception thrown from {nameof(SyslogReaderService)}.");
}
});
}
private void handleLine(object sender, DataReceivedEventArgs e)
{
_logger.LogDebug($"Posting new line to internal processing queue.");
_targetQueue.Post(e.Data);
}
}
}
If there is an obvious issue with my code above that would fix this issue then a simple solution to that would be great, however I am thinking that I may need to take a different approach.
First let me say my understanding of syslog is not very strong, but from what I understand you can configure syslog to forward events to a 'syslog server'. So I am thinking that I could configure syslog to send events to a localhost port and have my app respond to those events. With this solution I think I can bypass the uncertainty of tailing the file and also dealing with the rolling of the file if I just incrementally read and evaluate which lines are new. So if there isn't an obvious issue with my solution then how do I configure syslog to send copies of events to my c# application running locally?
I ended up configuring syslog to send events to a local udp port and then listened on that port on the c# side. Tailing the file was not the way to go.