Parse `key1=value1 key2=value2` in Kusto

Question

I'm running Cilium inside an Azure Kubernetes Cluster and want to parse the cilium log messages in the Azure Log Analytics. The log messages have a format like

key1=value1 key2=value2 key3="if the value contains spaces, it's wrapped in quotation marks"

For example:

level=info msg="Identity of endpoint changed" containerID=a4566a3e5f datapathPolicyRevision=0 

I couldn't find a matching parse_xxx method in the docs (e.g. https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/parsecsvfunction ). Is there a possibility to write a custom function to parse this kind of log messages?

Answer 1

With the help of Slavik N, I came with a query that works for me:

let containerIds = KubePodInventory
| where Namespace startswith "cilium"
| distinct ContainerID
| summarize make_set(ContainerID);
ContainerLog
| where ContainerID in (containerIds)
| extend KeyValuePairs = array_concat(
    extract_all("([a-zA-Z0-9_-]+)=([^ \"]+)", LogEntry),
    extract_all("([a-zA-Z0-9_]+)=\"([^\"]+)\"", LogEntry))
| mv-apply KeyValuePairs on 
(
    extend p = pack(tostring(KeyValuePairs[0]), tostring(KeyValuePairs[1]))
    | summarize JSONKeyValuePairs=parse_json(make_bag(p))
)
| project TimeGenerated, Level=JSONKeyValuePairs.level, Message=JSONKeyValuePairs.msg, PodName=JSONKeyValuePairs.k8sPodName, Reason=JSONKeyValuePairs.reason, Controller=JSONKeyValuePairs.controller, ContainerID=JSONKeyValuePairs.containerID, Labels=JSONKeyValuePairs.labels, Raw=LogEntry