Storing a lot of graphic data in memory (principle)

I'm developing simple application for 2D drawing with ability to use layers. I tried different platforms (.NET, Java, QT...) and I always run into the same problem. I can't understand how to store all graphic layers/images/bitmaps in memory efficiently, especially when canvas (each object that stores graphic data) size is big (Like 3000x3000 pixels or so). I suppose there's no other way but to store every layer as image/bitmap object (or any kind of color-data-array that still will consume alot of memory).

What I have already tried:

Storing all layer-objects directly in memory. I get fast access to these objects but if canvas size is too big and there are many layer-objects, program often crashes due to memory usage overflow.
Storing only "active" layer-object in memory, while keeping all other layers cached on hard drive. But I still need all layer-objects to be accessed every time user draws a stroke to rebuild new final "visible" image. And when I access all cached layer-objects from hard drive, it's really slow, because I need to allocate all of them in memory again one by one and kill them after.
Compress graphic data in all layer-objects and store it in memory. But again, when I have to decompress graphic data one by one every time I need to rebuild final visible image, it's really slow too.

So what is the correct way to store alot of graphic objects? How do popular software like Photoshop can handle many large layers and still work fast? Does anyone know?

---- S O L U T I O N ----

Okay! Thanks to evilruff, I solved this. Looks like when I read a small needed slice of layer from cached file, memory allocation proceeds much faster than when I try to allocate the whole layer.

Here's what I did:

On layer-object initialization, I create new file on hard drive, filled with zero-bytes (transparent pixels). That way layer's graphic data is not stored in memory.
When I need to get a bitmap/image slice from layer-object, I pass slice region coordinates. Method in layer-object creates new empty bitmap/image and substitutes it's "bits" with cached file's data, starting with the correct stride and position according to the region.
After all manipulations with bitmap/image are done, I pass it back to another layer-object's method that updates cached file with bitmap/image's new "bits". After that I just flush bitmap/image from memory.

Works very fast and there's only one visible bitmap/image allocated in memory.

Bad side: cached file for every layer still takes hard drive's space. But nowdays that's not critical when everyone has large hard-drives.

Solution

Well it depends.. What is the nature of your drawings? Are they initially vector based (like maps, diagrams what so ever) or you work with set of raster images which you put as a set of layers?