Search code examples
amazon-s3amazon-ec2amazon-cloudfrontfuses3fs

Very slow connection between EC2 and S3. Is it normal?


enter image description hereI am trying to setup CDN for my website.

So, this is what I did:

  1. Created S3 bucket
  2. Created CloudFront distribution based on this test bucket
  3. Mounted my test s3 bucket on my ec2 instance using following command:

    /usr/bin/s3fs -o allow_other this-is-test-bucket /mnt/s3drivetest/

(I used this tutorial to install Fuse 2.9.1 and s3fs 1.61 on CentOS 6.3 instance- http://www.itisopen.net/2011/12/CentOS_and_s3fs/)

Now when I try to copy files from my ec2 to this mounted drive "/mnt/s3drivetest/" it takes forever to copy all files. 40MB folder with lots of small images and css files is copied in 2 hours. Approximately 5 files per second.

cp -Rfv /var/www/website.com/media/ /mnt/s3drivetest/ 

Is it normal? I tried to copy the same folder to my /tmp/ directory and it took 2 seconds.

Also, what are these "0 byte" files that are created with the same name as folders? Please see screenshot.

INFO: EC2 instance is located in "region=us-east-1" and S3 bucket location is "US Standard".

EC2 Instance OS: CentOS 6.3


Solution

  • This is most likely not a problem with S3 but with the s3fs filesystem implementation being slow. There are additional details in this question.