Search code examples
cfilestreamlibc

Extending libc's fmemopen stream structure to own the buffer and deallocate it on fclose


In libc there's an extension mechanism for implementing custom FILE* streams which allows custom open/read/seek/close callback to be attached to a custom struct.

Some docs I found on custom stream and cookie implementation:

I'm looking for a primer to extend fmemopen(...) behavior so that the buffer passed at the stream creation will get deallocated at fclose(...), but extend it in a way to reuse existing fmemopen functionality and hopefully in a portable way with respect do different libc's. Is it possible? How would one go about doing this?

In a way, my question is whether it's possible to "inherit" from opaque struct/pointer returned by fmemopen and "override"/"wrap" the close callback with a version which also free()s the buffer pointer passed in.

Thanks!


UPD: Currently I implemented this functionality by tracking all pointers FILE* produced by fmemopen in an array and keeping corresponding reference of allocated buffer. And then I have a custom close function which goes over this array and deallocates the sidekick buffer. But it seems that this libc cookie mechanism can make this more elegant.

Is it possible to claw back the buffer pointer from the fmemopen-produced FILE* in a "portable" way (glibc/musl libc)?


Solution

  • I'm looking for a primer to extend fmemopen(...) behavior so that the buffer passed at the stream creation will get deallocated at fclose(...), but extend it in a way to reuse existing fmemopen functionality and hopefully in a portable way with respect do different libc's. Is it possible?

    If you let POSIX fmemopen() allocate the buffer for you (by passing a null pointer as the first argument) then that buffer will be deallocated automatically at fclose(). No extension needed. Of course, this is useful only when you open the stream for updating, because the only way to get out the data you write is to reposition the stream and read the data back via I/O functions.

    If you're looking instead to make the stream take over management of a pre-allocated buffer that you provide to it then my first suggestion is simply don't. Obscuring the management of allocated memory that way is poor form. If you're planning on losing the pointer to the buffer before the stream is closed, then let fmemopen() manage the buffer from beginning to end, as already described. If you're not planning on losing the buffer pointer during the lifetime of the stream, then you simply should not be trying to attach deallocation automagically to file closure. Deallocate explicitly. If you feel the need, then write a function that handles both the closure and deallocation. It sounds like you're already doing something much like this, so good on you. I promise that this will save you grief down the road.

    If you nevertheless insist on implementing a custom stream that has the memory-management behavior you describe, then it appears that MUSL has adopted GNU's fopencookie() extension, so you should be able to use that both with Glibc and with MUSL (do verify). I am not aware of it being available from other C standard library implementations. It is possible that you could this way wrap a stream provided by fmemopen(), but you cannot extend in the sense of directly using or calling the four I/O functions associated with streams provided by fmemopen().