Search code examples
jsonlogstashlogstash-grok

how to write filter section in logstash for json formatted nested file


I have below log file.

{
    "level":"error",
    "msg":"err:ERR_AUTH_PORTAL_USER,tenantid:7,reason:out of access period,code:EP05,username:Che,venueid:10,devicemac:##-##-##-##-##-##,apmac:##-##-##-##-##-##,ssid:Jetwig,timestamp:Jan 22 2018 09:05:31 UTC",
    "time":"2018-01-22T14:35:31+05:30"
}

I want to filter them according to msg: err:ERR_AUTH_PORTAL_USER using logstash grok filter. How can I do this?

This is what I have tried so far:

input {
    file {
        type => vampserror
        path => "/home/ampsErrorLog/getError/*"
                start_position => "beginning"
        }
}

filter {
    grok {
        patterns_dir => ["./patterns"] 
        match => {  "message" => "%{LOGLEVEL:level} %{MESSAGE:msg} %{TIMESTAMP:timestamp}" }
    }
}

if "ERR_AUTH_PORTAL_USER" in [msg] {

}

output {
    stdout { codec => rubydebug }
}

Solution

  • input { 
        file{
        type=>"json"
        path=>"logfile location"
        start_position => "beginning"
        }
    
    }
    
    filter {     
    
          json{
            source => "message"
    
    
         }   
    }  
    
    output {
    
        if[msg][err]=="ERR_AUTH_PORTAL_USER"{
    
        stdout { codec => rubydebug }
    
        mongodb {
        collection => "error"
        database => "dev"
        uri => "mongodb://localhost:27017"
        isodate => true
    
         }
    
        }
    
    file{
            path => "/tmp/output.txt"
        }  
    
    }
    

    add this to your conf file

     mongodb {
        collection => "error"
        database => "dev"
        uri => "mongodb://localhost:27017"
        isodate => true
    
         }
    

    is optional