Search code examples
cfilesystemsramdisk

Create Large File in /dev/shm in parallel: performance

I need to create a large file (several gbytes) in /dev/shm. which is a RAMdisk on Linux machines. This must be as fast as possible. I'm currently using 5 separate POSIX threads in C to create sequential 100 mbyte segments, then I have another thread which concatenates them onto the main output file.

This is fast, but I want to go faster. Can I eliminate the concatenate thread? Is there any way to open a file, and have each thread write it's 100 mbyte segment to the correct place in the final output file?

Solution

  • The fastest way to do this is to just call truncate() or ftruncate() to expand the file to the size you want.

    You can then mmap() the file into the process's memory space, and have each thread write its section into the mapped area.