I am configuring flume to stream weather data, i have written flume agent as given below,
WeatherAgent.sources= Weather
WeatherAgent.sinks = hdfs-write
WeatherAgent.channels= memory-channel
WeatherAgent.sources.Weather.type = weather
WeatherAgent.sources.Weather.bind = api.openweathermap.org/data/2.5/forecast/city?id=524901&APPID=*********
WeatherAgent.sources.Weather.port = 11111
WeatherAgent.sinks.hdfs-write.type = hdfs
WeatherAgent.sinks.hdfs-write.hdfs.path = hdfs://localhost:8020/user/hadoop/flume
WeatherAgent.sinks.hdfs-write.rollInterval = 30
WeatherAgent.sinks.hdfs-write.hdfs.writeFormat=Text
WeatherAgent.sinks.hdfs-write.hdfs.fileType=DataStream
WeatherAgent.channels.memoryChannel.type = memory
WeatherAgent.channels.memoryChannel.capacity=10000
WeatherAgent.sources.Weather.channels=memoryChannel
WeatherAgent.sinks.hdfs-write.channel=memoryChannel
i am getting error as
weather agent does not contain any valid channels making it as invalid.
i am new to flume,i dont know which all parameters to be addded in agent config file,i even dont know written agent is correct or wrong,please help me to solve this. if any Help links are there to write custom or new agents please post.
thanks in advance
Suyog
it is type mismatching. In your configuration file you mentioned
WeatherAgent.channels= memory-channel
but you are using memoryChannel instead of memory-channel
WeatherAgent.channels.memoryChannel.type = memory
WeatherAgent.channels.memoryChannel.capacity=10000
WeatherAgent.sources.Weather.channels=memoryChannel