Search code examples
elasticsearchlogstashelastic-stacklogstash-groklogstash-configuration

How can I make indexing json file by using logstash?


I try to make index my json file like below. I have to write a grok expression . But I could not do that? can you help me?

{"level":"Information","ClientIP":"10.201.21.188","Test":"10.210.21.188"}
{"level":"Information","ClientIP":"10.202.21.187","Test":"10.220.21.188"}
{"level":"Information","ClientIP":"10.203.21.186","Test":"10.230.21.188"}
{"level":"Information","ClientIP":"10.204.21.185","Test":"10.240.21.188"}

My logstash.conf is below :

input {
  file {
    type => "json"
    path => ["C:/logs/test-20170933.json"]
    start_position => "beginning"
  }
}
filter {
  grok { 
         match => [ "message","%{WORD:level}   I HAVE TO WRITE OTHER ELEMENTS  BUT HOW????"]
  }
  json {
          source => "message"
       }
}
output {
	 stdout {
    codec => rubydebug
    }
    elasticsearch {
        hosts => [ "localhost:9200" ]
        index => "logstash-%{+YYYY.MM.dd}"
    }
}

I guess that we need grok expression to achive that. Also I am open for new creative solution for that.


Solution

  • You don't need to grok anything, your file input simply needs a JSON codec and you're good to go:

    input {
      file {
        type => "json"
        path => ["C:/logs/test-20170933.json"]
        start_position => "beginning"
        codec => "json"                  <-- add this
      }
    }
    filter {
    }
    output {
        stdout {
            codec => rubydebug
        }
        elasticsearch {
            hosts => [ "localhost:9200" ]
            index => "logstash-%{+YYYY.MM.dd}"
        }
    }