I am using boost::iostreams::mapped_file_source to create a memory mapped files. In excess of 1024. To my surprise when I have created around 1024 memory mapped files my program throws an exception stating there are too many files open. After some research I found that Ubuntu uses a max file size per processes of 1024 (found from ulimit -n). Unfortunately, I need all of the files to be open at the same time. Does anyone know a way around this? Is it possible to make the files not count towards the limit someway? I was thinking of trying to combine them into a single file; however, I would like to avoid that if possible due to the performance. And I would also like to not modify the operating system by changing the value. Any points in the correct direction are much appreciated!
Why do you need many mapped files open? That seems very inefficient. Maybe you can map (regions of) a single large file?
Q. I was thinking of trying to combine them into a single file; however, I would like to avoid that if possible due to the performance
That's ... nonsense. The performance could basically only increase.
One particular thing to keep in mind is to align the different regions inside your "big mapped file" to multiple of your memory page/disk block size. 4k should be a nice starter for this coarse alignment.