Race condition against Google Photos API

Question

I'm developing this CLI to upload images to Google Photos. The CLI creates several go routines to upload files in parallel 1. Once it's uploaded, the same routine adds it to an Album. These albums are created if they didn't exist before 2.

Due to the concurrency and the fact that Google Photos API allows the creation of two Albums with the same name, I'm having duplicated album names that I'd like to avoid.

The GetOrCreateAlbumByName() 3 doesn't ensure that the Album is unique. It basically ask if an album exists with the same name and if not, it will create a new one. But this function could be called in parallel so two Album with the same name could be created. Besides it's implementing a mutex, I'm observing duplicates.

How would yo recommend to deal with that?

1. Create a Worker (like a microservice) that deal with album creation. Each go routine will be blocked until the album is created. It will remove concurrency on the Album creation part.
2. Maintain an album cache and use it to check if the album has been already created. In that case, race condition could happen too, but with less probability.
3. Using 1 and 2 together.
4. Other, please specify.

I have concerns about 1, 2 and 3... that's why I would like to know how you would deal with that.

Thanks in advance

Answer 1

You use Client.GetOrCreateAlbumByName() to get or create albums. This method is safe for concurrent use, and it uses a mutex internally to serialize calls, which ensures duplicates are not created.

However, you are creating and using multiple clients here, which means concurrent calls to all of them is not serialized, only concurrent calls to individual clients.

So the solution to either use a single client, then all calls will be serialized, or if that's not possible, you have to serialize calls to all clients, e.g. using a single mutex or other means.

An example how to serialize calls to all clients with a mutex:

var mu sync.Mutex

func GetOrCreateAlbumByName(c *gphotos.Client, name string) (*photoslibrary.Album, error) {
    mu.Lock()
    defer mu.Unlock()
    return c.GetOrCreateAlbumByName(name)
}

And wherever you use client.GetOrCreateAlbumByName(name), replace it with the call:

GetOrCreateAlbumByName(client, name)