Search code examples
javascriptnode.jsamazon-web-servicesamazon-s3aws-sdk-nodejs

How to download a file from Amazon S3 bucket in node.js synchronously


I have to download multiple files from S3 bucket using node.js. For that I have to write a for loop & call the s3.getObject(param) method to download. After the files are downloaded I have to merge their contents.

I have written like this:

var fileContentList = new ArrayList();

for(i=0; i<fileNameList.length i++){
    s3.getObject({ Bucket: "my-bucket", Key: fileNameList.get(i) }, function (error, data) {
    if (error != null) {
      alert("Failed to retrieve an object: " + error);
    } else {
      alert("Loaded " + data.ContentLength + " bytes");
      fileContentList.add(data.Body.toString());
    }
  }
);
}

//Do merging with the fileContentList.

But as s3.getObject is an asynchronous call the current thread moves on & nothing gets added to the fileContentList while I am doing the merging.

How can I solve the problem? Any idea?
Is their any synchronous method in aws-sdk to download file?


Solution

  • I have solved using this. Though I have not tried the answers of Alexander,Lena & Sébastian I believe each of the answers mentioned by them would also work in this case. Many many thanks to them for their quick reply:

    Async.eachSeries(casCustomersList, function (customerName, next){
            if(casCustomersList.length>0 && customerName != customerId) {
    
                var paramToAws = {
                    Bucket: bucketName,
                    Key: folderPath +'applicationContext-security-' + customerName + '.xml'      //file name
                };
                AWSFileAccessManager.downloadFile(paramToAws, function (error, result) {
                    if (error) {
                        next(error);
                    } else {
                        customerApplicationContext.add(result.Body.toString());
                        next();
                    }
    
                });
            } else{
                next();
            }
    
        }, function(err) {
           //Write the rest of your logic here to process synchronously as it is the callback function
    
        }