Getting this error while running. I am trying add connectiion between the mongodb and elasticsearch. Adding data through logstash for continuous monitoring. And then we will add to Kibana. But encountered this errror, Dont know whether its conf file formatting issue or what?
# bin/logstash -f /etc/logstash/conf.d/mongodata.conf --config.debug
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/usr/share/logstash/logstash-core/lib/jars/jruby-complete-9.2.11.1.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2020-06-17 11:12:25.705 [main] runner - --config.debug was specified, but log.level was not set to 'debug'! No config info will be logged.
[WARN ] 2020-06-17 11:12:25.937 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2020-06-17 11:12:25.939 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"7.7.0"}
[ERROR] 2020-06-17 11:12:26.495 [Converge PipelineAction::Create<main>] agent - Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"{\" at line 2, column 15 (byte 23) after input {\n \"uri\" ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:58:in `compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:66:in `compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:28:in `block in compile_sources'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:27:in `compile_sources'", "org/logstash/execution/AbstractPipelineExt.java:181:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:67:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342:in `block in converge_state'"]}
[INFO ] 2020-06-17 11:12:26.649 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2020-06-17 11:12:31.694 [LogStash::Runner] runner - Logstash shut down.Here is my conf file:
input {
"uri" => "mongodb://liquid.nvidia.com:27017/"
"placeholder_db_dir" => "/opt/logstash-mongodb/"
"placeholder_db_name" => "logstash_sqlite.db"
"collection" => "global_queue_list"
"batch_size" => 5000
}
filter {
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
"action" => "index"
"index" => "mongo_log_data"
"hosts" => ["localhost:9200"]
}
}You're missing the mongodb input plugin (see here)
input {
mongodb {
"uri" => "mongodb://liquid.nvidia.com:27017/"
"placeholder_db_dir" => "/opt/logstash-mongodb/"
"placeholder_db_name" => "logstash_sqlite.db"
"collection" => "global_queue_list"
"batch_size" => 5000
}
}