I am a novice in the world of LogStash. Just started to learn it. I tried to create a config file called Unhealthy_data.config using data from a similarly named csv file.
The contents of my config file are as below: -
input{
file{
path => "D:/01_Users/LogStash/Unhealthy.csv"
start_position => "beginning"
}
filter{
csv{
separator => ","
columns => ["cluster_name","unhealthy_nodes","userid","applicationid","queue","application_type","impact_host","cluster_utilization","queue_utilization","running_containers","running_memory","elapsed_time","tech_datestamp"]
}
}
output{
elasticsearch{
hosts =>"http://localhost:9200"
index => "unhealthy"
document_type => "unhealthy_data"
}
stdout{}
}
}
The last column "tech_datestamp" is a Date column.
I am unable to load the data and get the error as below: -
C:\ELK\logstash-7.9.1\bin>logstash -f C:\ELK\LogStash\UnhealthyData.config
Sending Logstash logs to C:/ELK/logstash-7.9.1/logs which is now configured via
log4j2.properties
[2020-11-28T07:33:35,924][INFO ][logstash.runner ] Starting Logstash {"
logstash.version"=>"7.9.1", "jruby.version"=>"jruby 9.2.13.0 (2.5.7) 2020-08-03
9a89c94bcc Java HotSpot(TM) 64-Bit Server VM 25.271-b09 on 1.8.0_271-b09 +indy +
jit [mswin32-x86_64]"}
[2020-11-28T07:33:36,158][WARN ][logstash.config.source.multilocal] Ignoring the
'pipelines.yml' file because modules or command line options are specified
[2020-11-28T07:33:37,058][ERROR][logstash.agent ] Failed to execute ac
tion {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"L
ogStash::ConfigurationError", :message=>"Expected one of [A-Za-z0-9_-], [ \\t\\r
\\n], \"#\", \"=>\" at line 7, column 6 (byte 131) after input{\r\n\tfile{\r\n\t
\tpath => \"D:/01_Users/LogStash/Unhealthy.csv\"\r\n\t\tstart_posi
tion => \"beginning\"\r\n\t}\r\n\tfilter{\r\n\t\tcsv", :backtrace=>["C:/ELK/logs
tash-7.9.1/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "
org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logst
ash/execution/JavaBasePipelineExt.java:69:in `initialize'", "C:/ELK/logstash-7.9
.1/logstash-core/lib/logstash/java_pipeline.rb:44:in `initialize'", "C:/ELK/logs
tash-7.9.1/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'"
, "C:/ELK/logstash-7.9.1/logstash-core/lib/logstash/agent.rb:357:in `block in co
nverge_state'"]}
[2020-11-28T07:33:37,355][INFO ][logstash.agent ] Successfully started
Logstash API endpoint {:port=>9600}
[2020-11-28T07:33:42,306][INFO ][logstash.runner ] Logstash shut down.
[2020-11-28T07:33:42,328][ERROR][org.logstash.Logstash ] java.lang.IllegalSta
teException: Logstash stopped processing because of an error: (SystemExit) exit
Request
You have not closed your input section before opening your filter section. As a result, the logstash configuration compiler interpets the csv filter as a csv input
Try moving the final } to after the filter section.