I am in the process of laying down the requirements for a photography community site. An important feature to investigate would be allowing more fotos/account than rival sites around my country's internet. What are the possibilities out there?
Should I go for something like amazon S3, or is there anything that offers more image-related features? I am mostly interested in low price per GB (storage and transfer out).
I used to work for a social networking website that hosts billions of images and we evaluated S3. Conclusion was that it is too expensive for heavy-traffic sites. The storage itself is pretty cheap, but the costs for accessing the content on S3 add up quickly. That makes S3 more suitable for applications like online backups. In my view, cost is the main con.
On the other hand, this is only a concern once your site gets large. The biggest advantages of S3 are that you don't have to worry about scalability and that it's pretty easy to set up and then forget about it because it just works. Many medium sized services use S3 with great success.
The solution we went for is an array of dedicated servers that host the images and also run webservers (don't use Apache, use webservers optimized for static content such as lighttpd or nginx), and in front of those, use a CDN (content delivery network, such as akamai or panther express). You will typically get high hit rates (depending on the access patterns of your site), so the end users will get most files directly out of the CDN and not cause any load on your servers (except for the first time a file is accessed). Thus you might be fine with just one server and a mirror for a while. As you scale, the challenges become how to distribute your images across the farm, how to manage redundancy etc.
I assume that time-to-market also plays a role. In that respect, a good strategy might be to start with S3 and be up-and-running quickly. Later on you can still migrate to a more sophisticated solution. In that case, make sure management keeps this in mind. Non-tech people tend to believe once a functionality works, you never have to touch it again. And be aware that migrating a lot of data takes time. When we changed our photo architecture, the copy jobs ran for months.