This is a design doubt am facing, I have a collection of 1500 images which are to be displayed on an asp.net page, the images to be displayed differ from one page to another, the count of these images will increase in the time to come,
a.) is it a good idea to have the images on the database, but the round trip time to fetch the images from the database might be high.
b.) is it good to have all the images on a directory, and have a virtual file system over it, and the application will access the images from the directory
Do we have in particular any design strategy in a traditional database for fetching images with the least round trip time, does any solution other than usage of a traditional database exists?
Edit 1: Each image is replaced by its new entry for every 12 hours, so having them on the database might not be a good idea as far I can think of, but how better will it be to use a data-store and index these images?
Edit 2: Yes, we are planning to run the application on a cluster. And if we try to use a datastore (if it is a good option to go with) then is it compatible with C# & ASP.NET?
ps: I use SQL Server to store these images.
All of the previous comments are really good... In the absence of very specific requirements we have to make broad generalizations to illustrate your options. Here are a few examples.
If raw speed is what you need, then flat files are a clear winner. Whether your using Apache or IIS, they're both optimized to serve static file-based content very fast. High performance sites all know this, and will store much of their content with dynamic handling in mind but will then "publish" select pieces of their dynamic content; in static versions; to their web farm on a periodic or event-driven basis. Although it requires a bit of orchestration, it can be done cheaply and can really reduce the load of your database server, backend network, etc. As a simple example, publish to a folder with the root of that structure being dynamically assessed. when you're ready to publish updates, write a new folder, and then change the root path. No down time, cheap, and easy. On a related note, pulling all this information from a backend store will require you to load these things into memory. This will ultimately translate to more time in Garbage Collection and consequently will mean your application is slower, even if you're using multi-processor/core gardening.
If you need fine-grained control over how images are organized/exposed, then folders may not be the most appropriate. If for example, you need to organize an individual users images and you need to track a lot of meta data around the images then a database may be a good fit. With that said, your database team will probably hate you for it, because this presents a number of challenges from a database management perspective. Additionally, if you're using an ORM you may have some trouble making this work and may find your memory footprint grows to unacceptable levels due to hidden proxy objects, second-level caching, etc. This can all be mitigated so just watch out and make sure you profile your application. With that said, a structured store (like a DB) is more ideal for this use case.
Considering security... Depending on what these images represent, flat files inevitably lead to concerns about canonicalization attacks, brute-force enumeration of browsable folder structures, replaying cookies, urls, viewstate, etc. If you're using a custom Role-based or Claims-based security model you may find using flat files becomes somewhat painful since you'll have to map filesystem security constraints to logical/contextual security constraints. Issues like these often lead me to favoring a structured store.
The aformentioned cache idea is a good one and can help create some middle-ground with respect to how often you actually hit your database, although it will not help with concerns related to memory consumption, GC, etc... You could employ built-in caching mechanisms although a Cache/Grid that supports backing stores would be much better if you can afford it (Ex. NCache, ScaleOut, etc.). These provide nice scalability/redundency, and can also be used to offload storage of session state, viewstate, and a lot more.
Hope this helps.