Search code examples
meteoramazon-s3collectionfs

File download from API to Meteor server and upload to S3

I am sending a request from my Meteor server to download a file via an API. I then want to upload that file to S3. I keep getting the following "NoSuchKey: The specified key does not exist." I initially thought it was maybe a problem with my AcessKey/SecretKey form AWS but after googling this for a while the only examples I could find of other people getting this error is when trying to download a file from S3.

Setting up cfs:s3

var imageStore = new FS.Store.S3("images", {

  accessKeyId: "MyAcessKeyId", //required if environment variables are not set
  secretAccessKey: "MySecretAcessKey", //required if environment variables are not set
  bucket: "BucketName", //required

});

Images = new FS.Collection("images", {
  stores: [imageStore]
});

Start file transfer from API and upload to S3 

client.get_result(id, Meteor.bindEnvironment(function(err, result){ //result is the download stream and id specifies which file to download.
  if (err !== null){
    return;
  } 
   var file = new FS.File(result);
   Images.insert(file, function (err, fileObj) {
    if (err){
      console.log(err);
    }

  });
}));

Note: I was getting the following error so I added Meteor.bindEnvironment. "Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment."

Node.js example from API Documentation

client.get_result(id, function(err, result){
        if (err != null) {            
            return;
        }

        file.writeFile(path.join('public', path.join('results', filename)), result, 'binary');
    });
Solution

  • What ended up fixing the problem for me was moving part of the setup to the lib folder. Although I tried several different ways I was unable to get it to execute entirely on the server. It looks like the documentation was updated recently which states everything a bit more clearly. If you follow this setup it should eliminate the error. See the section titled Client, Server, and S3 credentials

    https://github.com/CollectionFS/Meteor-CollectionFS/tree/master/packages/s3

    Note: Make sure not to place you secret key is not in you lib folder as this is accessible from the client.