Basically, I want to compress a file in a pthread thread using gzip. The first solution that pops up in mind and on Google is to call system().
What does the stackoverflow community suggest?
Shall I use system() in a pthread?
Or shall I myself just fork and exec in pthread? But since pthread is a thread, is it advisable to do a fork() and exec() in pthread thread?
Or what is the better approach than the above?
You shouldn't use system for this, but not because it's expensive. (For any file that is worth bothering to compress, the overhead of any technique for invoking a background gzip compression is negligible relative to the cost of doing the compression itself.) The reason you shouldn't use system is, system invokes a shell, and that means you have to worry about quoting the arguments. If you use fork and execvp instead, you don't have to worry about quoting anything.
The problems associated with mixing fork and wait with threads are real, but they are tractable. If your OS has posix_spawn, it will take care of some of those problems for you. I don't normally recommend posix_spawn because code that uses it is, in general, harder to maintain than code that uses fork, but for this application it should be OK. I cannot fit a comprehensive guide to mixing fork and wait with threads into this answer box.
An alternative you should consider is compressing the data in the thread that would be waiting for the gzip process, using zlib. This avoids the problems of mixing fork and wait with threads, but it adds a library dependency to your program, which may not be as convenient as relying on an external gzip executable.