Splunk search events

Question

For events format:

LogMessage: { [-]
 message: { [-]
   data: { [-]
     ConnectorRequest details:
       request: { [-]
         application: bond
         items: [ [-]
           { [-]
             naturalKey: ************4206
             system: debitCard
           }
         ]
       }
       url: https://token-service.apps.cloud.com/v1/tokenization
     }
   }
}

I can use search:

index="0010-pcf-abc-service*" | search "LogMessage.message.data.ConnectorRequest details.request.items{}.naturalKey"="************4206"

and it returns results with specific naturalKey value.

Similar search for all events with specific url value:

index="0010-pcf-abc-service*" | search "LogMessage.message.data.ConnectorRequest details.url=https://token-service.apps.cloud.com/v1/tokenization"

returns zero results but in reality, there are thousands of such events in the index.

Why is that?

Answer 1

There could be a few problems at play here.

Firstly, your second search structure does not match your first one. The | search command needs to be structured as follows:

| search "complex key"="complex value"

Not:

| search "complex key=complex value"

Note the difference in quotation-marks.

Secondly, you might be hitting field extraction limits. To address this, you can either force a JSON field extraction using spath or modify your limits.conf.

Spath example:

index="0010-pcf-abc-service*"
| spath "LogMessage.message.data.ConnectorRequest details.url"
| search "LogMessage.message.data.ConnectorRequest details.url"="https://token-service.apps.cloud.com/v1/tokenization"

You can read more about limits.conf in Splunk's documentation.