Parse LogQL substring as JSON

Question

Is it possible to apply | json to a substring of a log line?

Example Log line

2023-07-17T19:24:51: {"msg":"LOGIN_SUCCESS","meta":{"userNameOrEmail":"[email protected]","user":{"id":1611561848468,"user_group_id":1530117853159,"username":null,"primary_email":"[email protected]","num_additional_groups":1},"user_groups":[{"id":1110467853159,"name":"Demo"},{"id":1657113131838,"name":"Agency"}]}}

Current LogQL

{job="pm2", filename=~".*server-out.*.log"} |~ `{"msg":.*` 

I would like to parse JSON substring so I can group by the message, IDs, etc

The bash equivalent of what I'm trying to do is:

echo -n '2023-07-17T19:24:51: {"msg":"LOGIN_SUCCESS","meta":{"userNameOrEmail":"[email protected]","user":{"id":1611561848468,"user_group_id":1530117853159,"username":null,"primary_email":"[email protected]","num_additional_groups":1},"user_groups":[{"id":1110467853159,"name":"Demo"},{"id":1657113131838,"name":"Agency"}]}}' \
  | grep -Eo '{"msg":.*' \
  | jq

Answer 1

Yes you can do that:

1. Extract json part of your log into label,
2. use line_format to output extracted part
3. output result into json

{job="pm2", filename=~".*server-out.*.log"}
 | regexp `(?P<json_part>\{"msg":.*)`
 | line_format {{.json_part}}
 | json