stream create --name TailTest --definition "tail --name=/Users/name/Documents/SpringXD/PoC/input/Try.txt --outputType=text/plain --lines=1 |file --name=output --dir=/Users/name/Documents/SpringXD/PoC/output --mode=APPEND" --deploy
Here I see that whether I give --lines=1 or 2 or 0 as soon as there is an additional line entered in Try.txt and it is saved, it is reading entire Try.txt and dumping it into output file. Again when I add a line to Try.txt it does the same reads entire file content rather than just last line and appends it at the end of the previous content in output, any idea what could be going wrong here, why does tail does not read only last 1 (or specified n) lines I do put enter at the end of each line in input file?
It sounds like you are replacing the file when you say "and it is saved".
Tail follows an existing file and, yes, it expects a newline to terminate a new message.
I just ran a test and it works fine...
xd:>stream create --name foo --definition "tail --lines=1 --name=/Users/foo/Documents/foo.txt | log" --deploy
logs the last line of the file,
23:15:58,925 INFO SimpleAsyncTaskExecutor-1 sink.foo - sdsd
then
$ echo foo >> ~/Documents/foo.txt
(which appends foo\n to the file)
results in
23:17:07,744 INFO SimpleAsyncTaskExecutor-1 sink.foo - foo
The lines=1 only applies to an existing file; if you replace the file with a new one, the whole file is used - this is like log rotation.