What are the performance characteristics of memory mapping the same file multiple times? Will the OS reuse/cache between the mappings or will it read in the file multiple times into different parts of the memory?
i.e. if I read and write to a memory mapped file from two different processes will it go through the disk or will they communicate in memory? If I read from a memory mapped file from two different process alternatively thread, will they read from the same memory?
You can use MAP_PRIVATE in order to get a private copy-on-write mapping of the underlying file. You can use MAP_SHARED in order to get a view of the underlying file. I do not know what happens if you have the same region mapped in MAP_PRIVATE mappings and MAP_SHARED mappings at the same time, but I suspect writes to either mapping cause a copy.
In order to sync with disk, you want to use msync. On most systems, multiple MAP_SHARED mappings will give you a piece of memory, nominally backed by disk, that is shared among processes. I'm not sure whether this behaviour is specified or merely a happy consequence of the typical implementation.