How to flatten JSON for a generic type in Go

Question

I'm trying to implement HAL in Go, just to see if I can. This means that I've got a HAL type that is generic over the payload, and also contains the _links:

type HAL[T any] struct {
    Payload T
    Links   Linkset `json:"_links,omitempty"`
}

In the HAL spec, the payload is actually at the top level and not nested inside it - like, e.g. Siren would be. So that means given the following:

type TestPayload struct {
    Name   string `json:"name"`
    Answer int    `json:"answer"`
}

    hal := HAL[TestPayload]{
        Payload: TestPayload{
            Name:   "Graham",
            Answer: 42,
        },
        Links: Linkset{
            "self": {
                {Href: "/"},
            },
        },
    }

The resulting JSON should be:

{
    "name": "Graham",
    "answer": 42,
    "_links": {
      "self": {"href": "/"}
    }
}

But I can't work out a good way to get this JSON marshalling to work.

I've seen suggestions of embedding the payload as an anonymous member, which works great if it's not generic. Unfortunately, you can't embed generic types in that way so that's a non-starter.

I probably could write a MarshalJSON method that will do the job, but I'm wondering if there's any standard way to achieve this instead?

I've got a Playground link with this working code to see if it helps: https://go.dev/play/p/lorK5Wv-Tri

Cheers

Answer 1

Yes, unfortunately you can't embed the type parameter T. I'll also argue that in the general case you shouldn't attempt to flatten the output JSON. By constraining T with any, you are admitting literally any type, however not all types have fields to promote into your HAL struct.

This is semantically inconsistent.

If you attempt to embed a type with no fields, the output JSON will be different. Using the solution with reflect.StructOf as an example, nothing stops me from instantiating HAL[[]int]{ Payload: []int{1,2,3}, Links: ... }, in which case the output would be:

{"X":[1,2,3],"Links":{"self":{"href":"/"}}}

This makes your JSON serialization change with the types used to instantiate T, which is not easy to spot for someone who reads your code. The code is less predictable, and you are effectively working against the generalization that type parameters provide.

Using the named field Payload T is just better, as:

• the output JSON is always (for most intents and purposes) consistent with the actual struct
• unmarshalling also keeps a predictable behavior
• scalability of the code is not an issue, as you don't have to repeat all of the fields of HAL to build an anonymous struct

OTOH, if your requirements are precisely to marshal structs as flattened and everything else with a key (as it might be the case with HAL types), at the very least make it obvious by checking reflect.ValueOf(hal.Payload).Kind() == reflect.Struct in the MarshalJSON implementation, and provide a default case for whatever else T could be. Will have to be repeated in JSONUnmarshal.

Here is a solution with reflection that works when T is not a struct and scales when you add more fields to the main struct:

// necessary to marshal HAL without causing infinite loop
// can't declare inside the method due to a current limitation with Go generics
type tmp[T any] HAL[T]

func (h HAL[T]) MarshalJSON() ([]byte, error) {
    // examine Payload, if it isn't a struct, i.e. no embeddable fields, marshal normally
    v := reflect.ValueOf(h.Payload)
    if v.Kind() == reflect.Pointer || v.Kind() == reflect.Interface {
        v = v.Elem()
    }
    if v.Kind() != reflect.Struct {
        return json.Marshal(tmp[T](h))
    }

    // flatten all fields into a map
    m := make(map[string]any)
    // flatten Payload first
    for i := 0; i < v.NumField(); i++ {
        key := jsonkey(v.Type().Field(i))
        m[key] = v.Field(i).Interface()
    }
    // flatten the other fields
    w := reflect.ValueOf(h)
    // start at 1 to skip the Payload field
    for i := 1; i < w.NumField(); i++ {
        key := jsonkey(w.Type().Field(i))
        m[key] = w.Field(i).Interface()
    }
    return json.Marshal(m)
}

func jsonkey(field reflect.StructField) string {
    // trickery to get the json tag without omitempty and whatnot
    tag := field.Tag.Get("json")
    tag, _, _ = strings.Cut(tag, ",")
    if tag == "" {
        tag = field.Name
    }
    return tag
}

With HAL[TestPayload] or HAL[*TestPayload] it outputs:

{"answer":42,"name":"Graham","_links":{"self":{"href":"/"}}}

With HAL[[]int] it outputs:

{"Payload":[1,2,3],"_links":{"self":{"href":"/"}}}

Playground: https://go.dev/play/p/bWGXWj_rC5F