Removing Json object name using Logtash

Question

I'm trying to parse the following json using logstash

   {
    "payload": "{\"class\":\"OpenLoyalty\\\\Component\\\\Account\\\\Domain\\\\Event\\\\PointsTransferHasBeenExpired\",\"payload\":{\"accountId\":\"1cbd42b8-1b1f-4a13-a28f-eefeb73a64b0\",\"pointsTransferId\":\"e82c96cf-32a3-43bd-9034-4df343e5f111\"}}"
}

using this filter :

    input {
        jdbc {
            jdbc_connection_string => "jdbc:postgresql://localhost:5432/openloyalty"
            jdbc_user => "openloyalty"
            jdbc_password => "openloyalty"
            jdbc_driver_library => "C:\logstash\postgresql-42.2.12.jar"
            jdbc_driver_class => "org.postgresql.Driver"
            statement => "SELECT payload from events where events.id=130;"
        }
    }
    filter {
          json {
            source => "payload"
            remove_field => ["class"]
          }

    }
    output {
        stdout { codec => json_lines }
    }

I get this output:

    {
    "@version": "1",
    "@timestamp": "2020-05-12T11:27:15.097Z",
    "payload": {
        "accountId": "1cbd42b8-1b1f-4a13-a28f-eefeb73a64b0",
        "pointsTransferId": "e82c96cf-32a3-43bd-9034-4df343e5f111"
    }
   }

however, what I'm looking for is like this:

{
"@version": "1",
"@timestamp": "2020-05-12T11:27:15.097Z",
 {
    "accountId": "1cbd42b8-1b1f-4a13-a28f-eefeb73a64b0",
    "pointsTransferId": "e82c96cf-32a3-43bd-9034-4df343e5f111"
 }
}

how can I remove the "payload" name from my Json object?

Answer 1

You can use the mutate filter with the following config after your json filter.

mutate {
    rename => {
        "[payload][accountId]" => "accountId"
        "[payload][pointsTransferId]" => "pointsTransferId"
    }
    remove_field => ["[payload]"]
}

This will rename your fields and remove the empty payload json object after the rename.