I am working with a file API that only provides a const char* filename interface (accepting - for stdout) when writing files. I would instead like the output to be written into memory, so I can then pass the data elsewhere.
I can use dup2 to redirect stdout to an arbitrary file descriptor and something like fmemopen/open_memstream as my sink. However, the memory stream functions expect a size, which -- in my case -- I don't know in advance and could be arbitrarily large.
The data I need to access, however, does have a fixed length and offset within what's being produced (e.g., out of 1MB, I need the 64KB starting at 384KB, etc.). As such, is there a way to set up a circular buffer with fmemopen/open_memstream that just keeps being rewritten until reaching the offset in question? (I realise this is inefficient, but there's no ability to seek.)
Or is this the wrong approach? I've read a bit about memory mapped files and that seems to be similar to what I'm trying to achieve, but it's not something I know much about...
EDIT Just to be clear, I cannot write anything to disk.
Using dup2 redirect stdout to a pipe and call the API with - to instruct it to use standard output. Then read from the pipe the data generated by the API, filter it and store it in a memory region.
If the pipe capacity is not enough, you will need two threads to make this approach work.
One will be running the API call generating data and putting it into the pipe.
The other thread, will take the data from the pipe, check the offset and store the data in memory when the target offset is reached. But keep reading from the pipe until EOF so that the other thread can complete the API call and finish gracefully.