Search code examples
hadoopflume

I want to read a log file in flume which is on the different server

I want to read a log file from different server in flume which is up and running on some different server.......so for doing so how can I achive this by changing my flume-conf.properties file.......what should i write in the configuration file of flume to achieve this...

a1.sources = AspectJ
a1.channels = memoryChannel
a1.sinks = kafkaSink
a1.sources.AspectJ.type = com.flume.MySource
a1.sources.AspectJ.command = tail -F /tmp/data/Log.txt

for achiving this what should I write in place of a1.sources.AspectJ.command = tail -F /tmp/data/Log.txt

Solution

  • I believe what you want to ask is that, if Flume is setup on host 'F' and your log files exists on host 'L', how will you configure flume to read log files from host 'L', correct ?

    If so, then you need to setup Flume on host 'L' and not on 'F'. Setup flume on the same host where the log files are and setup the Sink to point to Kafka topic.