How does nxlog track the line number?

Question

In nxlog config I have these params set:

SavePos True
ReadFromLast True

When removing lines from a log file (this should never happen) nxlog ships the entire log file. Is this related to how nxlog tracks the line number?

To recreate:

1. I stop the nxlog service
2. Delete the nxlog cache (just to make sure im starting fresh)
3. Right now the log folder I've configured nxlog to watch is empty
4. I add a new log file to the folder
5. nxlog ships the log file
6. I open the log file and add a few lines
7. nxlog ships those lines
8. I delete those new lines I just added
9. nxlog ships the entire log file

Answer 1

NXLog and generally other log shippers are designed to deal with append-only log files. When you delete lines from the log file it sees that the file size is less. Under the append-only assumption this can only mean that the file was replaced/rotated and the current file is a new one that needs to be fully read.

Also note that when you edit a log file in a text editor the editor will usually replace the file with a new one even if you only append data to the end. This is not equivalent to echo test >> test.log.

If you want to transfer all kinds of changes in files you should use rsync or other tools.