Search code examples
memorymemcachedmemory-mapped-filesmemory-mapping

How Reduce Disk read write overhead?


I have one website mainly composed on javascript. I hosted it on IIS. This website request for the images from the particular folder on hard disk and display them to end user. The request of image are very frequent and fast.

Is there any way to reduce this overhead of disk read operation ?

I heard about memory mapping, where portion of hard disk can be mapped and it will be used as the primary memory. Can somebody tell me if I am wrong or right, if I am right what are steps to do this. If I am wrong , is there any other solution for this ?


Solution

  • While memory mapping is viable idea, I would suggest using memcached. It runs as a distinct process, permits horizontal scaling and is tried and tested and in active deployment at some of the most demanding website. A well implemented memcached server can reduce disk access significantly.

    It also has bindings for many languages including those over the internet. I assume you want a solution for Java (Your most tags relate to that language). Read this article for installation and other admin tasks. It also has code samples (for Java) that you can start of with.

    In pseudocode terms, what you need to do, when you receive a request for, lets say, Moon.jpeg is:

    String key = md5_hash(Moon.jpeg); /* Or some other key generation mechanism */
    IF key IN memcached
       SUPPLY FROM memcached /* No disk access */
    ELSE
       READ Moon.jpeg FROM DISK
       STORE IN memcached ASSOCIATED WITH key
       SUPPLY
    END
    

    This is a very crude algorithm, you can read more about cache algorithms in this Wiki article.