We have photo uploading functionality in the app which works for a lot of people, but some people get the following error:
"{
"responseData": "Metadata part is too large."
}
"
when trying to upload a file.
I use Google Cloud Storage using FormData() to upload a file through POST using signed policy document.
This is how I issue the policy using nodejs latest cloud storage library (5.18.1):
const gcsClient = new Storage({
projectId: EnvConfig.gcpProjectId,
credentials: getGoogleCloudStorageCredentials ()
});
const policyOpts = {
equals: ['$Content-Type', contentType],
expires: getTodaysDateSecondsAdded(900),
contentLengthRange: {
min: 0,
max: 20000000 // 20MB
}
};
const [{ base64, signature, string }] = await gcsClient
.bucket(EnvConfig.gcpUploadBucket)
.file(fileName)
.getSignedPolicy(policyOpts);
and this is how it gets uploaded:
const data = new FormData();
data.append('policy', policy64);
data.append('signature', policySignature);
data.append('file', {uri: sourceUrl, type: contentType});
data.append('key', fileName);
data.append('GoogleAccessId', googleAccessId);
data.append('Content-Type', contentType);
const CancelToken = axios.CancelToken;
const source = CancelToken.source();
try {
await axios.post(uploadUrl, data);
} catch (e) {
if (axios.isCancel(e)) {
throw new Error(`axios upload image failed due to timeout`);
}
throw e;
}
It is confusing to me what metadata part is being referenced to? Are we talking about the object metadata that's part of cloud storage concept? If yes, I have no control over it, I don't add any metadata anywhere? Or are we talking about the image metadata itself?
Is this just a bad error and it actually means that file size is too large?
According to this:
"There is a maximum combined size limit for all custom metadata keys and values of 8 KiB per object." - https://cloud.google.com/storage/quotas
So my current hypothesis is that somewhere a magic object metadata is generated that is over 8KB, is there some undocumented part of logic in cloud storage that automatically pulls custom metadata from POST request?
I saw bunch of potential duplicates but there's no answer out there:
Uploading File to GCS - Metadata part is too large
Is there a limit to the length of metadata values in Google Cloud Storage?
I have figured out after months of debugging,
basically when attaching file to form data, you also need to specify name: otherwise it will fail for bigger files >4MB+.
before (kept failing):
data.append('file', { uri: assetUri, type: contentType});
after:
data.append('file', { uri: assetUri, type: contentType, name: 'dummy.jpg' });
If you read react-native source code, you'll see the "name" will force emiting filename= in the multipart/formdata request