Search code examples
jsonpowershellperformancefiltering

How to get key and value pairs from converted JSON file


I converted in PowerShell a JSON file into a variable.

$JSON = Get-Content -Path "$LogFile" -Raw | ConvertFrom-Json

Sample (Object):

Id                  : Directory_825efe20-2c90-4b90-885f-589a38360cb9
Category            : ApplicationManagement
CorrelationId       : 825efe20-2c90-4b90-885f-589a38360cb9
Result              : success
ResultReason        : 
ActivityDisplayName : Update service principal
ActivityDateTime    : 20.09.2023 08:00:05
LoggedByService     : Core Directory
OperationType       : Update
InitiatedBy         : @{User=; App=}
TargetResources     : {@{Id=14fa3ae9-6dc9-4267-a681-368aa59ffbc8; DisplayName=ODS API; Type=ServicePrincipal; UserPrincipalName=; GroupType=; ModifiedProperties=System.Object[]}}
AdditionalDetails   : {@{Key=User-Agent; Value=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.62}, @{Key=AppId; Value=563e60f7-4012-491a-b10b-493afb3f2154}}

What is the best way (regarding performance) to extract a specific key/value pair for example the "User-Agent" from the AdditionalDetails property?

Thank you for your assistance!


Solution

  • Note: See the bottom section for sample JSON that the solutions can be tested with.

    A concise solution that performs reasonably well, using the intrinsic .Where() method:

    $objectOfInterest = 
      $JSON.AdditionalDetails.Where({ $_.Key -eq 'User-Agent' }, 'First')[0]
    

    Note the need for [0], due to .Where() always returning a collection of results (even though by definition there is at most one in this case).


    The following is likely faster, but more verbose and less conceptually direct, using a foreach statement:

    $objectOfInterest = 
      foreach ($obj in $JSON.AdditionalDetails) { 
        if ($obj.Key -eq 'User-Agent') { $obj; break }
      }
    

    The slowest - but arguably most PowerShell-idiomatic - option is to use the Where-Object cmdlet:

    $objectOfInterest = 
      $JSON.AdditionalDetails | Where-Object Key -eq User-Agent
    

    Note the use of simplified syntax, i.e. Key -eq User-Agent in lieu of { $_.Key -eq 'User-Agent' }

    Note that this will invariably process all input objects, because Where-Object offers no way to stop filtering once a match has been found. To do so, additional work would be required.

    • This surprising omission - especially given that Where-Object's expression-mode counterpart, .Where() does have this feature, as shown above ('First') - is the subject of GitHub issue #13834.

    As for why this is slowest:

    • The PowerShell pipeline (|), due to its object-by-object handoff between the commands involved, invariably introduces overhead.

      • However, note that the pipeline is an elegant, core feature of PowerShell that is generally worth taking advantage of, and only worth avoiding if performance requires it.
    • This overhead alone is not dramatic, but exacerbated by the currently inefficient implementation of the Where-Object cmdlet (as well as ForEach-Object), as of PowerShell (Core) 7.3.x

    • Finally, as noted, the fact that Where-Object (and ForEach-Object) invariably processes all input, with no ability to (gracefully) stop the pipeline once a match has been found, introduces overhead.

      • Introducing a -First switch, as discussed in the aforementioned GitHub issue #13834, would solve that specific problem.

      • However, there's also a need for a general mechanism to gracefully stop a pipeline from user code, as discussed in GitHub issue #3821.


    Sample JSON that the solutions above can be tested with (the basic structure matches the one implied by the for-display representation of the parsed-from-JSON object shown in your question):

    # Note: Better to call this variable $fromJSON, for instance.
    $JSON = @'
    {
      "foo": "bar",
      "AdditionalDetails": [
        {
          "Key": "AppId",
          "Value": "563e60f7-4012-491a-b10b-493afb3f2154"
        },
        {
          "Key": "User-Agent",
          "Value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.62"
        }
      ]
    }
    '@ | ConvertFrom-Json