I've got an express app that I'm hosting as a Google Cloud Function but it doesn't work on the Cloud the same as it works locally.
From my NextJS application, I'm sending a FileList to the endpoint by adding it to a new instance of a FormData. The way I'm adding it is by iterating through the state that stores the FileList and appending each file to the formData.
const [files, setFiles] = useState<FileList | undefined>();
const formData = new FormData();
for(let i = 0; i < files.length; i++){
formData.append('pdfFile', files[i]);
}
I'm then taking this formData and sending it as the body of a fetch request to the Cloud Function endpoint
fetch('https://[function_uri]/extract', {method: 'POST', body: formData})
.then((res) => res.json())
.then((res) => console.log('Response: ', res))
.catch((error) => console.error(error))
Here is how I have my express app setup
const express = require('express');
const pdfParse = require('pdf-parse');
const cors = require('cors');
const multer = require('multer');
const upload = multer();
const app = express();
app.use(cors());
app.post('/extract', upload.array('pdfFile'), async (req, res) => {
res.set('Access-Control-Allow-Origin', '*');
if (!req.files) {
res.status(400);
res.end();
}
const files = req.files;
const response = [];
if (files.length) {
for (let i = 0; i < files.length; i++) {
const buffer = Buffer.from(files[i].buffer);
await pdfParse(buffer).then((result) => {
response.push({
name: files[i].originalname,
text: result.text,
});
});
}
} else {
const buffer = Buffer.from(files.buffer);
await pdfParse(buffer).then((result) => {
response.push({
name: files.originalname,
text: result.text,
});
});
}
res.json({ response });
});
app.listen(9090, () => console.log('Server running on port 9090.'));
module.exports = {
app,
};
I'm using multer to get the files and when running the express app locally, everything works perfectly, however, once I deploy the code to the Google Cloud platform, req.files is now an empty array instead of being populated with the data I expected after running locally and I get errors thrown because the data is now undefined. I'm honestly not sure whether or not I'm sending data wrong, or if I'm handling it wrong on the server. This is my first time using Google Cloud Functions so I've been researching blindly at this point.
The following is an example of what I expected to get from req.files
[
{
fieldname: 'pdfFile',
originalname: 'FileName.pdf',
encoding: '7bit',
mimetype: 'application/pdf',
buffer: <Buffer 27 50 44 46 2d 31 2e 35 0d 0a 25 c5 b5 b5 b5 0d 0a 31 20 30 20 6a 62 6a 0d 0a 3c 3c 2f 54 79 70 65 2f 43 21 74 61 7c 6f 67 2f 50 61 67 65 73 20 32 20 ... 41169 more bytes>,
size: 50283
}
]
After a lot more searching, I found an article that helped me solve this problem. Turns out multer doesn't work with Google Cloud Functions so I used busboy instead. The article linked below worked well, however, when it came to actually getting the data of my files I had to add another block of code that the article didn't seem to cover.
busboy.on('file', (fieldname, file, { filename }) => {
file.on('data', (data) => {
filesData.push({
filename,
data,
});
console.log(`File ${filename} got ${data.length} bytes.`);
}).on('close', () => {
console.log(`File ${filename} done.`);
});
});
When busboy detects a file, each file has a .on('data') function that we can use to get the data and do what we need to do with it. In my case I saved it to an array to be used later in my function.
This solution is how I got my fix, I hope it helps!