Blobstore reader won't read large images

Question

I am trying to read images from the Blobstore from the Go server side code. But if the image is large (as in: 0.5MB - 1.7MB) a large portion of the byte buffer becomes 0 which breaks the image.

The image works if I use serveUrl, but this is not an option for me in this case.

My first thought was that there was a size limit to the read, found this:

In addition to systemwide safety quotas, a limit applies specifically to the use of the Blobstore: the maximum size of Blobstore data that can be read by the application with one API call is 32 megabytes.

The images I read are nowhere near 32MB.

Function I use to read from the Blobstore:

func BlobAsBase64(c appengine.Context, blobKey string) (*blobstore.BlobInfo, string, error) {
    info, err := blobstore.Stat(c, appengine.BlobKey(blobKey))
    if err != nil {
        return nil, "", err
    }

    imageBuffer := make([]byte, info.Size)
    reader := blobstore.NewReader(c, appengine.BlobKey(blobKey))
    if _, err = reader.Read(imageBuffer); err != nil {
        return nil, "", err
    }

    imageBase64 := base64.StdEncoding.EncodeToString(imageBuffer)

    return info, imageBase64, nil
}

What is the reason that my images become broken when I read them from the Blobstore?

Answer 1

I conjecture the reason this isn't working is because the reader.Read method is returning before it has filled the buffer. See the contract for io.Reader

Read reads up to len(p) bytes into p. It returns the number of bytes read (0 <= n <= len(p)) and any error encountered. Even if Read returns n < len(p), it may use all of p as scratch space during the call. If some data is available but not len(p) bytes, Read conventionally returns what is available instead of waiting for more.

Note the last sentence in particular.

Instead of reader.Read(imageBuffer) try using ioutil.ReadAll which should fix your problem.