copying s3 bucket to another aws account with CarrierWave Ruby gem

Question

I'm trying to copy some contents from a bucket to a bucket in a different aws account. I begin by loading uploader objects into a hash. Then I attempted to connect to the other bucket and save the assets using the credentials for that bucket.

    task :product_color_images => :environment do
  CarrierWave.configure do |c|
    c.fog_credentials = {
      :provider               => 'AWS',
      :aws_access_key_id      => ENV['COPY_FROM_AWS_KEY_ID'],
      :aws_secret_access_key  => ENV['COPY_FROM_AWS_KEY']
    }

    c.fog_directory = 'orig-bucket' # bucket copied from
  end

  image_storage = {}

  ProductImage.all.each do |image|
    puts 'storing product image'
    image_storage[image.id] = image.image
  end

  CarrierWave.configure do |c|
    c.reset_config
    c.fog_credentials = {
      :provider               => 'AWS',
      :aws_access_key_id      => ENV['COPY_TO_AWS_KEY_ID'],
      :aws_secret_access_key  => ENV['COPY_TO_AWS_KEY']
    }

    c.fog_directory = 'target-bucket' # bucket copied to
  end

  image_storage.each do |k, v|
    image = ProductImage.find(k)
    image.image = v
    puts 'saving product image'
    image.save
  end
end

Trying to save a single image from one bucket to the other in a console reveals that the address of the target bucket is not used.

ruby-1.9.2-p290 :026 > image = ProductImage.find(197) 
ruby-1.9.2-p290 :027 > image.image = image_storage[197]
 => https://orig-bucket.s3.amazonaws.com/uploads/product_image/image/197/product_image.png 
ruby-1.9.2-p290 :028 > image.save
ruby-1.9.2-p290 :029 > image.image
 => https://orig-bucket.s3.amazonaws.com/uploads/product_image/image/197/product_image.png

Answer 1

Sometimes it so happens that the bucket will be given enough permissions also make sure you have enough permission given to the images so that you can actually download them.

I have a better solution for you, what you can do is install and configure your s3cmd and do a rsync between two buckets. That will do things faster then your ruby on rails.