DynamoDBEvent produces attribute value JSON

Question

I'm writing a lambda that takes in streamed data from a DynamoDB table. After parsing out the proper record, I'm trying to convert it to JSON. Currently, I'm doing this:

func LambdaHandler(ctx context.Context, request events.DynamoDBEvent) error {

    // ...

    // Not actual code, just for demonstration
    record = request.Records[0]

    data, err := events.NewMapAttribute(record.Change.NewImage).MarshalJSON()
    if err != nil {
        return err
    }

    // ...
}

The problem is that this produces a JSON payload that looks like this:

{
  "M": {
    "action": { "N": "0" },
    "expiration": { "N":"0" },
    "id": { "S": "trades|v4|2023-02-08" },
    "order": { "N":"22947407" },
    "price": { "N":"96.139" },
    "sort_key": { "S":"22947407" },
    "stop_limit": { "N":"0" },
    "stop_loss": { "N":"96.7" },
    "symbol": { "S":"CADJPY" },
    "take_profit": { "N":"94.83" },
    "type": { "N":"5" },
    "type_filling": { "N":"0" },
    "type_time": { "N":"0" },
    "volume": { "N":"1" }
  }
}

As you can see, this mimics the structure of the DynamoDB attribute value but this isn't what I want. Instead, I'm trying to generate a JSON payload that looks like this:

{
  "action": 0,
  "expiration": 0,
  "id": "trades|v4|2023-02-08",
  "order": 22947407, 
  "price": 96.139,
  "sort_key": "22947407",
  "stop_limit": 0,
  "stop_loss": 96.7,
  "symbol": "CADJPY",
  "take_profit": 94.83,
  "type": 5,
  "type_filling": 0,
  "type_time": 0,
  "volume": 1
}

Now, I can think of a couple ways to do that: hardcoding the values from record.Change.NewImage into a map[interface{}] and then marshalling that using json.Marshal, but the type of the payload I receive could be one of several different types. I could also use reflection to do the same thing, but I'd rather not spend the time debugging reflection code. Is there functionality available from Amazon to do this? It seems like there should be but I can't find anything.

Answer 1

I ended up writing a function that does more or less what I need it to. This will write numeric values as strings, but otherwise will generate the JSON payload I'm looking for:

// AttributesToJSON attempts to convert a mapping of DynamoDB attribute values to a properly-formatted JSON string
func AttributesToJSON(attrs map[string]events.DynamoDBAttributeValue) ([]byte, error) {

    // Attempt to map the DynamoDB attribute value mapping to a map[string]interface{}
    // If this fails then return an error
    keys := make([]string, 0)
    mapping, err := toJSONInner(attrs, keys...)
    if err != nil {
        return nil, err
    }

    // Attempt to convert this mapping to JSON and return the result
    return json.Marshal(mapping)
}

// Helper function that converts a struct to JSON field-mapping
func toJSONInner(attrs map[string]events.DynamoDBAttributeValue, keys ...string) (map[string]interface{}, error) {
    jsonStr := make(map[string]interface{})
    for key, attr := range attrs {

        // Attempt to convert the field to a JSON mapping; if this fails then return an error
        casted, err := toJSONField(attr, append(keys, key)...)
        if err != nil {
            return nil, err
        }

        // Set the field to its associated key in our mapping
        jsonStr[key] = casted
    }

    return jsonStr, nil
}

// Helper function that converts a specific DynamoDB attribute value to its JSON value equivalent
func toJSONField(attr events.DynamoDBAttributeValue, keys ...string) (interface{}, error) {
    attrType := attr.DataType()
    switch attrType {
    case events.DataTypeBinary:
        return attr.Binary(), nil
    case events.DataTypeBinarySet:
        return attr.BinarySet(), nil
    case events.DataTypeBoolean:
        return attr.Boolean(), nil
    case events.DataTypeList:

        // Get the list of items from the attribute value
        list := attr.List()

        // Attempt to convert each item in the list to a JSON mapping
        data := make([]interface{}, len(list))
        for i, item := range list {

            // Attempt to map the field to a JSON mapping; if this fails then return an error
            casted, err := toJSONField(item, keys...)
            if err != nil {
                return nil, err
            }

            // Set the value at this index to the mapping we generated
            data[i] = casted
        }

        // Return the list we created
        return data, nil
    case events.DataTypeMap:
        return toJSONInner(attr.Map(), keys...)
    case events.DataTypeNull:
        return nil, nil
    case events.DataTypeNumber:
        return attr.Number(), nil
    case events.DataTypeNumberSet:
        return attr.NumberSet(), nil
    case events.DataTypeString:
        return attr.String(), nil
    case events.DataTypeStringSet:
        return attr.StringSet(), nil
    default:
        return nil, fmt.Errorf("Attribute at %s had unknown attribute type of %d",
            strings.Join(keys, "."), attrType)
    }
}

This code works by iterating over each key and value in the top-level mapping, and converting the value to an interface{} and then converting the result to JSON. In this case, the interface{} could be a []byte, [][]byte, string, []string, bool, interface{} or map[string]interface{} depending on the type of the attribute value.