Search code examples
jmespath

How to insert a attribute with a dynamic value in many object or add an element dynamically in many array?


I want to add an attribute in many object(situated in an array) and this value will be get dynamically. I use the JSON below, and I already made a query to extract what I want. We will start with th result of this query.

First my entire JSON:

[  
   {  
      "Nature":"lol",
      "EV":"lol",
      "Moves":[  
         {  
"Move":"OHKOmove",
            "Max":100,
            "Min":15
         },
         {  
"Move":"cacaz",
            "Max":35,
            "Min":20
         }
      ]
   },
   {  
      "Nature":"loi",
      "EV":"lal",
      "Moves":[  
         {  
"Move":"caca1",
            "Max":100,
            "Min":3
         },
{  
"Move":"caca2",
            "Max":100,
            "Min":3
         }
      ]
   },
   {  
      "Nature":"loi2",
      "EV":"lal",
      "Moves":[  
         {  
"Move":"caca1",
            "Max":100,
            "Min":3
         },
{  
"Move":"caca2",
            "Max":100,
            "Min":3
         },
{  
"Move":"caca3",
            "Max":100,
            "Min":3
         }
      ]
   },
   {  
      "Nature":"loi3",
      "EV":"lil",
      "Moves":[  
         {  
"Move":"caca1",
            "Max":100,
            "Min":3
         },
{  
"Move":"caca2",
            "Max":100,
            "Min":3
         },
{  
"Move":"caca3",
            "Max":100,
            "Min":3
         }
      ]
   }
]

Then my query: [?(length(Moves[?Max == `100`]) > `1`)].{Nature: Nature, EV: EV, Moves: Moves[?Max == `100`].Move, MovesCount: length(Moves[?Max == `100`].Move)} | [@,{MaxMouvCount: max_by(@, &MovesCount).MovesCount}][]

And the result of my query give this:

JSON Format Example 1

[
 {
   "Nature": "loi",
   "EV": "lal",
   "Moves": [
     "caca1",
     "caca2"
   ],
   "MovesCount": 2
 },
 {
   "Nature": "loi2",
   "EV": "lal",
   "Moves": [
     "caca1",
     "caca2",
     "caca3"
   ],
   "MovesCount": 3
 },
 {
   "Nature": "loi3",
   "EV": "lil",
   "Moves": [
     "caca1",
     "caca2",
     "caca3"
   ],
   "MovesCount": 3
 },
 {
   "MaxMouvCount": 3
 }
]

The idea is to put the attribute "MaxMouvCount": 3 on each objects in the array and then delete it from the array to give a result like this:

JSON Format Example 2

[
  {
    "Nature": "loi",
    "EV": "lal",
    "Moves": [
      "caca1",
      "caca2"
    ],
    "MovesCount": 2,
    "MaxMouvCount": 3
  },
  {
    "Nature": "loi2",
    "EV": "lal",
    "Moves": [
      "caca1",
      "caca2",
      "caca3"
    ],
    "MovesCount": 3,
    "MaxMouvCount": 3
  },
  {
    "Nature": "loi3",
    "EV": "lil",
    "Moves": [
      "caca1",
      "caca2",
      "caca3"
    ],
    "MovesCount": 3,
    "MaxMouvCount": 3
  }
]

In the title I talk about array, in fact with .* after my query I can transform the object in array and maybe put more easier the value in each array(matching with objects) and retransform array into object with object constructor. But I don't know how to do it. Can you help me please or tell me at least if it's possible.

PS: I use only JMESPath so I don't want an answer with any other language which contains JMESPath code(like javascript(in my case) or python or something else)


Solution

  • Quick Answer (TL;DR)

    • Usually it is easy to painlessly transform JSON with JMESPath
      • One pain-free key is to utilize JSON structures that are specifically normalized for optimal use with JMESPath
      • One pain-free key is knowing when to use dictionary (aka objects // associative-arrays // mappings) name-value pairs in your JSON to make all parts of the JSON capable of unambiguous reference
    • Unfortunately, the goal in this specific question is not doable with standard JMESPath, because JMESPath lacks a token to refer to the JSON data root in the current-node context

    Detailed Answer

    Context

    • JMESPath query language
    • python 3.x using JMESPath 0.9.4 [but any JMESPath engine will do]

    Problem

    • Scenario:
      • DeveloperSObosskay972 wishes to transform JSON data from one representation to another
      • DeveloperSObosskay972 wants to rely solely on JMESPath expressions to complete the transformation
      • DeveloperSObosskay972 wants to refer to one part of the JSON structure in another part, to allow for dynamic cross-referencing of the datastructure

    Attempt 01 (not what we really want)

    • the "almost" solution that is not quite what we want
    • this code ...
    import jmespath
    vdata001aa = """<<json.load(JSON Format Example 1)>>"""
    vresult = jmespath.compile('@|[*].{"Nature":@.Nature,"EV":@.EV,"Moves":@.Moves,"MovesCount":@.MovesCount,"MaxMouvCount":`3`}').search(vdata001aa)
    pprint.pprint(vresult)
    
    • produces this result ...
    [{'EV': 'lal',
      'MaxMouvCount': 3,
      'Moves': ['caca1', 'caca2'],
      'MovesCount': 2,
      'Nature': 'loi'},
     {'EV': 'lal',
      'MaxMouvCount': 3,
      'Moves': ['caca1', 'caca2', 'caca3'],
      'MovesCount': 3,
      'Nature': 'loi2'},
     {'EV': 'lil',
      'MaxMouvCount': 3,
      'Moves': ['caca1', 'caca2', 'caca3'],
      'MovesCount': 3,
      'Nature': 'loi3'},
     {'EV': None,
      'MaxMouvCount': 3,
      'Moves': None,
      'MovesCount': None,
      'Nature': None}]
    
    • This is not what we want, because:
      • we had to hardwire the value 3 for MaxMouvCount which is technically "cheating"
        • it's cheating because we want a dynamic value, not a hard-wired value
      • this produces a superfluous element where every value except MaxMouvCount is null (python happens to call this None)
      • we only wanted three elements, not four

    Attempt 02 (not what we really want)

    • The reason why Attempt 01 does not work well is because the original JSON stucture is not well-normalized for JMESPath
    • In order to address this, we add dictionary name-value pairs to the original data
    • With this approach, we make every element of the JSON data a value attached to a dictionary key (aka javascript object name-value pairs)
      • Here we use the term dictionary which is known in other contexts as object or hash or associative array or mapping
      • We don't care about the terminology, so much as the ability to refer to all parts of the top-level JSON as name-value pairs
    Attempt 02 // Part 1 (reformat JSON Format Example 1)
    • reformat your original JSON JSON Format Example 1 so it looks like this instead
    • this reformatted JSON will make all parts of your data unambiguously addressible
    {
    "jsontop": {
        "settings_info": {
          "MaxMouvCount": 3
        },
        "nature_table": [
         {
           "Nature": "loi",
           "EV": "lal",
           "Moves": [
             "caca1",
             "caca2"
           ],
           "MovesCount": 2
         },
         {
           "Nature": "loi2",
           "EV": "lal",
           "Moves": [
             "caca1",
             "caca2",
             "caca3"
           ],
           "MovesCount": 3
         },
         {
           "Nature": "loi3",
           "EV": "lil",
           "Moves": [
             "caca1",
             "caca2",
             "caca3"
           ],
           "MovesCount": 3
         }
        ]
    }
    
    Attempt 02 // Part 2 (run the pain-free JMESPath query to get what you want)
    • this code ...
    import jmespath
    vdata001aa  = """<<json.load(**RE-NORMALIZED** JSON Format Example 1)>>"""
    vresult     = jmespath.compile('@|jsontop.nature_table[*].{"Nature":@.Nature,"EV":@.EV,"Moves":@.Moves,"MovesCount":@.MovesCount,"MaxMouvCount":jsontop.settings_info.MaxMouvCount}').search(vdata001aa)
    pprint.pprint(vresult)
    pass
    
    • produces this result ...
    [{'EV': 'lal',
      'MaxMouvCount': None,
      'Moves': ['caca1', 'caca2'],
      'MovesCount': 2,
      'Nature': 'loi'},
     {'EV': 'lal',
      'MaxMouvCount': None,
      'Moves': ['caca1', 'caca2', 'caca3'],
      'MovesCount': 3,
      'Nature': 'loi2'},
     {'EV': 'lil',
      'MaxMouvCount': None,
      'Moves': ['caca1', 'caca2', 'caca3'],
      'MovesCount': 3,
      'Nature': 'loi3'}]
    
    • this is not what we want, because we get None (aka null) where we expected 3

    Explanation

    • Attempt 02 would have worked, if JMESPath supported a token to refer to the JSON root object
    • An alternate query which would work is (if for example, the dollar-sign character worked as a reference to the JSON data root)
    import jmespath
    vdata001aa  = """<<json.load(**RE-NORMALIZED** JSON Format Example 1)>>"""
    vresult     = jmespath.compile('@|jsontop.nature_table[*].{"Nature":@.Nature,"EV":@.EV,"Moves":@.Moves,"MovesCount":@.MovesCount,"MaxMouvCount":$.jsontop.settings_info.MaxMouvCount}').search(vdata001aa)
    pprint.pprint(vresult)
    pass
    

    Conclusion

    • Attempt 01 and Attempt 02 show that the current (stable) version of JMESPath does not quite have the ability to meet the requirement
    • You will have to break out of JMESPath to get the desired dynamic value from your JSON and populate the reformatted data
    • Alternatively, you will have to add an extension to JMESPath itself, which is arguably less desirable than using the functionality of the hosting language