Unmarshaling dynamodb items returns same item

Question

I have a range loop in golang, that loops through a map of type.AttributeValues returned by dynamodb using the scan function.

I loop through this slice of maps, using the built in function provided by the golang aws-sdk-v2 package attributevalue.UnmarshalMap() what happens is I get a a return of the same item for the length of the slice.

code:

   type Users struct {
    User *string `dynamodbav:"user"` 
    Email *string `dynamodbav:"email"` 
    }

    user := Users{}
  
   for _ , dbItem := range dynamoResults.Items { 
    err = attributevalue.UnmarshalMap(dbItem, &user)
   }

// Gives the same pointer location for all 
Users[0].User 
Users[1].User
Users[2].User

I have also attempted appending users into a slice from the results

// this just gives a slice with the same items as above
 u := []Users{}
for _ , dbItem := range dynamoResults.Items { 
        err = attributevalue.UnmarshalMap(dbItem, &Users)
        u = append(u, Users)
       }

The solution I found that works in using attributevalue.UnmarshalListOfMaps() that works just by giving this function the slice of Items from dynamoDb.

My question really is to understand why using the two above solutions did not work as expected wouldn't looping through the dynamodb items and passing them to the function UnMarshal each item individually? It seems that is just continued to do the same item.

Answer 1

If we shift the problem slightly--from Dynamo DB unmarshaling to JSON unmarshaling--then we can run it on the Go Playground. In the sample code I see two things that work together to cause duplicate slice values:

1. The unmarshalling target is defined outside of the loop (on line 51), so the results are written to the same location on each loop iteration (line 54) when u1Out is unmarshaled.

2. The fields of the User1 type are *string so when the loop variable value is copied to the slice (line 55), the strings that User and Email point to are not copied. Only their pointer values are copied.

Putting these two together, each iteration will write the unmarshaled User and Email values to the same memory location. This is demonstrated by the first two lines of the output, which print the elements of u1Out.

The remaining lines of output show the results in the other three cases: when either the target variable is moved inside the loop (u1In and u2In) or the fields are changed to string (u2Out). In each of these cases, new locations for the unmarshaled value are created as expected and no duplicates are printed.

package main

import (
    "encoding/json"
    "fmt"
)

type (
    Users1 struct {
        User  *string
        Email *string
    }
    Users2 struct {
        User  string
        Email string
    }
)

func main() {
    dbItems := []string{
        `{"User":"userA","Email":"userA@example.com"}`,
        `{"User":"userB","Email":"userB@example.com"}`,
    }

    u1Out := withVarDeclaredOutsideLoop[Users1](dbItems)
    printUsers(u1Out)

    u1In := withVarDeclaredInsideLoop[Users1](dbItems)
    printUsers(u1In)

    u2Out := withVarDeclaredOutsideLoop[Users2](dbItems)
    printUsers(u2Out)

    u2In := withVarDeclaredInsideLoop[Users2](dbItems)
    printUsers(u2In)

    // Output
    // u[0] = {User:0xc000014130 Email:0xc000014140}
    // u[1] = {User:0xc000014130 Email:0xc000014140}
    // u[0] = {User:0xc0000141c0 Email:0xc0000141d0}
    // u[1] = {User:0xc000014210 Email:0xc000014220}
    // u[0] = {User:userA Email:userA@example.com}
    // u[1] = {User:userB Email:userB@example.com}
    // u[0] = {User:userA Email:userA@example.com}
    // u[1] = {User:userB Email:userB@example.com}
}

func withVarDeclaredOutsideLoop[T any](dbItems []string) []T {
    var t T
    u := []T{}
    for _, dbItem := range dbItems {
        json.Unmarshal([]byte(dbItem), &t)
        u = append(u, t)
    }
    return u
}

func withVarDeclaredInsideLoop[T any](dbItems []string) []T {
    u := []T{}
    for _, dbItem := range dbItems {
        var t T
        json.Unmarshal([]byte(dbItem), &t)
        u = append(u, t)
    }
    return u
}

func printUsers[T any](u []T) {
    for i, user := range u {
        fmt.Printf("u[%d] = %+v\n", i, user)
    }
}