uploading file with axios to cloudinary file service and getting uploading progress

Question

I have an react component which uploads files to cloudinary with axios through my own web api endpoint. Overall process goes: axios post request to localhost/api/files/upload then at this endpoint I send files to cloudinary with this code:

export async function POST(req: NextRequest) {
  try {
    const formData = await req.formData();
    const formDataEntryValues = Array.from(formData.values());
    for (const formDataEntryValue of formDataEntryValues) {
      if (typeof formDataEntryValue === "object" && "arrayBuffer" in formDataEntryValue) {
        const file = formDataEntryValue as unknown as Blob;

        if (file.size > config.maxFileSize) {
          return BadRequest(`Max file size is ${config.maxFileSizeAsString}`);
        }

        const buffer = Buffer.from(await file.arrayBuffer());
        const response = await uploadWrapper(buffer);

        return NextResponse.json(response, { status: 200 });
      }
    }
  } catch (error: any) {
    return NextResponse.json(error, { status: 500 });
  }
}

const uploadWrapper = async (buffer: Buffer) => {
  return new Promise(async (resolve, reject) => {
// Here I stream file with cloudinary's library
    const cloud_upload_stream = await cloudinary.v2.uploader.upload_stream({
      folder: config.storePath
    }, (error, result) => {
      if (error) {
        reject(error);
      } else {
        resolve(result);
      }
    });
    streamifier.createReadStream(buffer).pipe(cloud_upload_stream);
  })
}

There I send files with axios and receive axios uploadProgressEvent to display it on web page. I see that progress works fine when I upload file to api route, at 100% upload I need to wait more because file is streamed with uploader.upload_stream to cloudinary. How can I pass progress from uploader.upload_stream to axios uploadProgress event with through my api? So the uploadProgressEvent shows uploading to api route and from api route to cloudinary?

Answer 1

To pass the upload progress from the uploader.upload_stream to the Axios uploadProgress event, you can create a custom Readable stream that intercepts data as it's being uploaded to Cloudinary and emits events to update the progress. Here's how you can modify your code to achieve this:

export async function POST(req: NextRequest) {
  try {
    const formData = await req.formData();
    const formDataEntryValues = Array.from(formData.values());
    for (const formDataEntryValue of formDataEntryValues) {
      if (typeof formDataEntryValue === "object" && "arrayBuffer" in formDataEntryValue) {
        const file = formDataEntryValue as unknown as Blob;

        if (file.size > config.maxFileSize) {
          return BadRequest(`Max file size is ${config.maxFileSizeAsString}`);
        }

        const buffer = Buffer.from(await file.arrayBuffer());
        const response = await uploadWrapper(buffer, (progress) => {
          // You can pass the progress here to the Axios uploadProgress event
          req.respondWith(
            NextResponse.json({ progress }, { status: 200 })
          );
        });

        return NextResponse.json(response, { status: 200 });
      }
    }
  } catch (error: any) {
    return NextResponse.json(error, { status: 500 });
  }
}

const uploadWrapper = async (buffer: Buffer, onProgress: (progress: number) => void) => {
  return new Promise(async (resolve, reject) => {
    const cloud_upload_stream = await cloudinary.v2.uploader.upload_stream({
      folder: config.storePath
    }, (error, result) => {
      if (error) {
        reject(error);
      } else {
        resolve(result);
      }
    });

    let bytesUploaded = 0;
    let totalBytes = buffer.length;

    streamifier.createReadStream(buffer)
      .on('data', (chunk) => {
        bytesUploaded += chunk.length;
        const progress = (bytesUploaded / totalBytes) * 100;
        
        // Call the progress callback to send progress to Axios
        onProgress(progress);
      })
      .pipe(cloud_upload_stream);
  });
}

In this modified code, I've added an onProgress callback to the uploadWrapper function, which gets called during the upload process to calculate and send the progress to the Axios uploadProgress event via the req.respondWith method.

This way, you can track and pass the upload progress from your API route to the client, showing progress during both the upload to your API route and the subsequent upload to Cloudinary. Good luck.