How to use Logstash filter to convert into nested object for elasticsearch output?

Question

I have following event or row from JDBC input.

{"academic_session_id" : "as=1|dur=2015-16,as=2|dur=2016-17",
          "branch_id" : 1}

I want to convert or format it into following using logstash filters...

{"branch_id": 1,"sessions":[{"as":"1","dur":"2015-16"},{"as":"2","dur":"2016-17"}]}

If you can suggest any alternative to logstash. Note- I am using Elasticsearch 5.X version

Answer 1

Since this is a pretty customized manipulation of the data, I would use the ruby filter, and just write a script using the code setting to parse the data. Something like this would work:

filter {
  ruby {
    code => "
      academic_session = event.get('academic_session_id').split(',').map{|data| data.split('|')}
      sessions = academic_session.map do |arr|
      temp_hash = {}
        arr.each do |kv|
          k,v = kv.split('=')
          temp_hash[k] = v
        end
        temp_hash
      end
      event.set('sessions', sessions)
    "  
    remove_field => ['academic_session_id']
  }
}