How to save part of a TreeSet to file efficiently? and reload it? (Java question)

I'm working with a TreeSet to store some information, so that it is sorted according to some order.

When the TreeSet becomes very large (>1GB), I want to save the smallest elements in the TreeSet to a file, to free some RAM. Then later, when there is more free RAM, I want to be able to reload these elements into memory to process them.

My question is: is there some efficient way of storing part of a TreeSet to file and restoring them into memory later?

Note that when I reload the elements into memory, it could be part of a new TreeSet or into the same TreeSet.

Thanks for any idea about how to do this!

Solution

What are you using the TreeSet for? Do the contents change often? Are you trying to be efficient in terms of speed, or disk usage?

Reading and writing to a File is very slow compared to memory, and keeping the file and memory versions in sync might be challenging if they change often.

Maybe it makes sense to use a database. There are several lightweight databases such as derby and sqllite which can be embedded in your application. Databases are designed to worry about the memory vs. file issue, and if you have >1Gb of data, maybe it makes sense to organise it.