Search code examples
kqlazure-sentinel

KQL - return entries not matching IP from watchlist (query optimization)


I want to receive a high severity alert in Sentinel when a user is added to a defined "high severity" group (via watchlist), however, I want to omit any users that are connected to a Zscaler IP address. The query below is working, however, I'm not sure this is the neatest/most optimized logic. Is there a shorter/better way to write this? I'm only concerned about the lines beginning with asterisks (which are only added for clarity).

watchlist "aadgroups"

Group Severity
Prod Owners High
Prod Contributors High

watchlist "ZSIPs"

zscaler_ip location
165.225.0.0/23 Chicago
165.225.60.0/22 Chicago
165.225.56.0/22 Chicago
let HighSeverityGroups = (_GetWatchlist('aadgroups') | where severity == "High" | project group_name, severity);
let ZSIPs = (_GetWatchlist('zscaler_ip') | project zscaler_ip);
AuditLogs
| where ActivityDisplayName == "Add member to group"
| where parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)) has_any (HighSeverityGroups)
| extend InitiatedByActor = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend GroupName = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)))
| extend Actor_ipv4 = tostring(parse_json(tostring(InitiatedBy.user)).ipAddress)
| extend TargetUser = tostring(TargetResources[0].userPrincipalName)
| project-reorder TimeGenerated,SourceSystem,InitiatedBy,ActivityDisplayName,TargetUser,GroupName,InitiatedByActor,Actor_ipv4,Result
| where TargetUser <> ""
** | evaluate ipv4_lookup(ZSIPs, Actor_ipv4, zscaler_ip, return_unmatched = true)
** | where isempty(zscaler_ip)

Solution

  • A couple of things you can try to optimize the query:

    • This filter is quite costly: | where parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)) has_any (HighSeverityGroups) - if TargetResources will rarely have strings from HighSeverityGroups, then before this filter, you can add a much more efficient filter, that will filter out most of the records: | where TargetResources has_any (HighSeverityGroups) - this way, the heavy parsing will be done only on a small amount of records
    • You're parsing some of the data more than once, for example tostring(parse_json(tostring(InitiatedBy.user)) - instead, you need to use the extend operator to parse them only once, and then use later on in the query