hcl, json, go : how can I iterate JSON?

Question

Problem: I’m trying to iterate through JSON-content and present the result like key,value pairs.

I've written some code that read hcl-files, these are then decoded with hcldec.Decode, and the result is then converted to JSON. These hcl-files define source and target for the application like this:

source.hcl:

source json "namefile" {
  attr firstName  {
    type = "varchar"
    expr = "$.firstName"
    length = "30"
  }
  attr lastName  {
    type = "varchar"
    expr = "$.lastName"
    length = "40"
  }
  attr gender  {
    type = "varchar"
    expr = "$.gender"
    length = "10"
  }
  attr age {
    type = "varchar"
    expr = "$.age"
    length = "2"
  }
}

target.hcl

target table {
  cols firstName {
    name=source.json.namefile.attr.firstName.expr
    type=source.json.namefile.attr.firstName.type
    length=source.json.namefile.attr.firstName.length
  }
  cols lastName {
    name=source.json.namefile.attr.lastName.expr
    type=source.json.namefile.attr.lastName.type
    length=source.json.namefile.attr.lastName.length
  }
}

The decoding is done like this:

tspec := hcldec.ObjectSpec{
    "target": &hcldec.BlockMapSpec{
        TypeName:   "target",
        LabelNames: []string{"table"},
        Nested: hcldec.ObjectSpec{
            "cols": &hcldec.BlockMapSpec{
                TypeName:   "cols",
                LabelNames: []string{"name"},
                Nested: &hcldec.ObjectSpec{
                    "name": &hcldec.AttrSpec{
                        Name:     "name",
                        Type:     cty.String, //cty.List(cty.String),
                        Required: false,
                    },
                    "type": &hcldec.AttrSpec{
                        Name:     "type",
                        Type:     cty.String, //cty.List(cty.String),
                        Required: false,
                    },
                    "length": &hcldec.AttrSpec{
                        Name:     "length",
                        Type:     cty.String, //cty.List(cty.String),
                        Required: false,
                    },
                },
            },
        },
    },
}


targ, _ := hcldec.Decode(body, tspec, &hcl.EvalContext{
    Variables: map[string]cty.Value{
        "source": val.GetAttr("source"),
    },
    Functions: nil,
})

j := decodeCtyToJson(targ, true)
log.Debugf("targ -j (spec): %s", string(j))    // debug info

Where the decodeCtyToJson return []byte like this:

func decodeCtyToJson(value cty.Value, pretty bool) []byte {
    jsonified, err := ctyjson.Marshal(value, cty.DynamicPseudoType)
    if err != nil {
        log.Debugf("Error: #v", err)
        return nil
    }
    if pretty {
        return jsonPretty.Pretty(jsonified)
    }
    return jsonified
}

Now, when I'm trying to testprint the JSON-content I'm not getting what I'm looking for:

var result map[string]interface{}
json.Unmarshal(j, &result)
log.Debugf("result: %# v", result)

tgtfil := result["value"].(map[string]interface{})
log.Debugf("tgtfil: %v", tgtfil)

log.Debugf("len(tgtfil): %# v", len(tgtfil))
for key, value := range tgtfil {
    log.Debugf("key: %# v", key)
    log.Debugf("value: %# v", value)
}

I am trying to get key, value pairs. But I'm getting this (first the whole JSON pretty print as wanted, then I am trying to loop through the JSON):

DEBU[0000] targ -j (spec): {
  "value": {
    "target": {
      "table": {
        "cols": {
          "firstName": {
            "length": "30",
            "name": "$.firstName",
            "type": "varchar"
          },
          "lastName": {
            "length": "40",
            "name": "$.lastName",
            "type": "varchar"
          }
        }
      }
    }
  },
  "type": [
    "object", 
    {
      "target": [
        "map", 
        [
          "object", 
          {
            "cols": [
              "map", 
              [
                "object", 
                {
                  "length": "string",
                  "name": "string",
                  "type": "string"
                }
              ]
            ]
          }
        ]
      ]
    }
  ]
} 
DEBU[0000] result: map[string]interface {}{"type":[]interface {}{"object", map[string]interface {}{"target":[]interface {}{"map", []interface {}{"object", map[string]interface {}{"cols":[]interface {}{"map", []interface {}{"object", map[string]interface {}{"length":"string", "name":"string", "type":"string"}}}}}}}}, "value":map[string]interface {}{"target":map[string]interface {}{"table":map[string]interface {}{"cols":map[string]interface {}{"firstName":map[string]interface {}{"length":"30", "name":"$.firstName", "type":"varchar"}, "lastName":map[string]interface {}{"length":"40", "name":"$.lastName", "type":"varchar"}}}}}} 
DEBU[0000] tgtfil: map[target:map[table:map[cols:map[firstName:map[length:30 name:$.firstName type:varchar] lastName:map[length:40 name:$.lastName type:varchar]]]]] 
DEBU[0000] len(tgtfil):  1                              
DEBU[0000] key: "target"                                
DEBU[0000] value: map[string]interface {}{"table":map[string]interface {}{"cols":map[string]interface {}{"firstName":map[string]interface {}{"length":"30", "name":"$.firstName", "type":"varchar"}, "lastName":map[string]interface {}{"length":"40", "name":"$.lastName", "type":"varchar"}}}} 

Process finished with exit code 0

My aim here is to eventually be able to iterate through alle the attributes defined in the target.hcl (length, name and type for each cols in this case). Then generate DDL-code from this information and finally implement the DDL in e.g. Presto.

But as of now I’m not able to isolate this information.

Any pointers on how to do this is appreciated.

Thanks, /b

Answer 1

The solution for me was to create a struct for the target and not use the target spec.