Is there a way to filter out the nested JSON string out into separate fields in fluentd?
Current JSON:
{
Value1: "something",
Value2: "something",
Message:{
Value3: "Something",
Value3: "Something"
}
}
What I want (or something similar):
{
Value1: "something",
Value2: "something",
Message.Value3: "Something",
Message.Value3: "Something"
}
The JSON doesn't have to be flatten like in above example but I do want the values to be in their own separate fields(columns) when they reach elasticsearch. In other wards, I want to split the single long Message string into multiple fields contained within it.
Try this:
<source>
...
</source>
<filter myapp**>
@type parser
key_name Message
format multi_format
<pattern>
format json # try parsing json in the first place
</pattern>
<pattern>
format none # leave as is if this is not json (plaintext)
</pattern>
reserve_data true # keep the original Message field in case anything go wrong
</filter>
<match myapp**>
...
</match>
Multi-format parser: https://github.com/repeatedly/fluent-plugin-multi-format-parser