How to flatten out a nested json structure in go

Question

I am getting nested data from mongo and I want to flatten that out in a structure to store it in a csv file.

The data looks like this:

{
    "_id" : "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
    "data" : {
        "driver" : {
            "etaToStore" : 156
        },
        "createdAt" : 1532590052,
        "_id" : "07703a33-a3c3-4ad5-9e06-d05063474d8c"
    }
} 

And the structure I want to eventually get should be something like this

type EventStruct struct {
    Id                  string      `bson:"_id"`
    DataId              string      `bson:"data._id"`
    EtaToStore          string      `bson:"data.driver.etaToStore"`
    CreatedAt           int         `bson:"data.createdAt"`
}

This doesn't work, so following some SO answers I broke it down into multiple structures:

// Creating a structure for the inner struct that I will receive from the query
type DriverStruct struct {
    EtaToStore  int     `bson:"etaToStore"`
}

type DataStruct struct {
    Id          string `bson:"_id"`
    Driver      DriverStruct `bson:"driver"`
    CreatedAt   int `bson:"createdAt"`
}
// Flattenning out the structure & getting the fields we need only
type EventStruct struct {
    Id                  string      `bson:"_id"`
    Data                DataStruct  `bson:"data"`
}

This gets all the data from the Mongo query result but it's nested:

{
  "Id": "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
  "Data": {
     "Id": a33-a3c3-4ad5-9e06-d05063474d8c,
     "Driver": {
        "EtaToStore": 156
     },
     "CreatedAt": 1532590052
  }
}

What I want to end up with is:

{
  "Id": "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
  "DataId": "a33-a3c3-4ad5-9e06-d05063474d8c",
  "EtaToStore": 156,
  "CreatedAt": 1532590052
}

I'm sure there's an easy way to do this but I can't figure it out, help!

Answer 1

You can implement the json.Unmarshaler interface to add a custom method to unmarshal the json. Then in that method, you can use the nested struct format, but return the flattened one at the end.

func (es *EventStruct) UnmarshalJSON(data []byte) error {
    // define private models for the data format

    type driverInner struct {
        EtaToStore int `bson:"etaToStore"`
    }

    type dataInner struct {
        ID        string      `bson:"_id" json:"_id"`
        Driver    driverInner `bson:"driver"`
        CreatedAt int         `bson:"createdAt"`
    }

    type nestedEvent struct {
        ID   string    `bson:"_id"`
        Data dataInner `bson:"data"`
    }

    var ne nestedEvent

    if err := json.Unmarshal(data, &ne); err != nil {
        return err
    }

    // create the struct in desired format
    tmp := &EventStruct{
        ID:         ne.ID,
        DataID:     ne.Data.ID,
        EtaToStore: ne.Data.Driver.EtaToStore,
        CreatedAt:  ne.Data.CreatedAt,
    }

    // reassign the method receiver pointer
    // to store the values in the struct
    *es = *tmp
    return nil
}

Runnable example: https://play.golang.org/p/83VHShfE5rI