Search code examples
clinuxpiping

Indefinite Piping in C


I am not exactly sure why the waitpid() just hangs permanently. It is my assumption that the forks should terminate and return back to the parent process after the execvp() is done. This is not happening. If I comment out the waitpid(), the initial output is correct but the program begins to behave unexpectedly.

edit
- execFirst() Handles execution of the first command from a list
- execMid() Handles execution of any command in the list that is not the first or last command in the list
- execLast() Handles execution of the last command in the list

// Closes pipe
void closePipe(int *pPipe){
    close(pPipe[0]);
    close(pPipe[1]);

    return;
}

// Opens the read end of pPipe and closes the write end
// @Private
void readFromPipe(int *pPipe){
    dup2(pPipe[0],0);
    close(pPipe[1]);
    close(pPipe[0]);

    return;
}

// Opens the write end of pPipe and closes the read end
// @Private
void writeToPipe(int *pPipe){
    dup2(pPipe[1],1);
    close(pPipe[0]);
    close(pPipe[1]);

    return;
}
// @Private
void execLast(sSettings *pSettings_, sCommand *pCommand, int iPipe[]){
    readFromPipe(iPipe);
    _execvp(pSettings_, pCommand);
}

// @Private
void execMid(sSettings *pSettings_, sCommand *pCommand, int iPipe1[], int iPipe2[]){
    readFromPipe(iPipe1);
    writeToPipe(iPipe2);
    _execvp(pSettings_, pCommand);
}

// @Private
void execFirst(sSettings *pSettings_, sCommand *pCommand, int iPipe[]){
    writeToPipe(iPipe);
    _execvp(pSettings_, pCommand);
}

void execIndefDepthPipe(sSettings *pSettings_, sCommandList *pCommandList_){
    int iPipe1[2];
    int iPipe2[2];

    int iStatus, iProcessId, iNumCommands = pCommandList_->iSize;
    _pipe(pSettings_, iPipe1);
    _pipe(pSettings_, iPipe2);

    sCommand *pCommand;
    do{
        pCommand    = popHeadNode(pCommandList_);
        iProcessId  = _fork(pSettings_);

        if(iProcessId){ // If Parent
            // continue; // Do nothing
            waitpid(iProcessId, &iStatus, WUNTRACED); // It waits forever. I don't know why.
        }
        else { // Execute the commands as a child
            if(pCommand != NULL) {
                if(pCommandList_->iSize == (iNumCommands-1)) { // Exec First
                    //closePipe(iPipe2);
                    execFirst(pSettings_, pCommand, iPipe1);
                }
                else if(pCommandList_->iSize == 0) { // Exec Last
                    closePipe(iPipe1);
                    execLast(pSettings_, pCommand, iPipe2);
                }
                else { // Exec mid
                    execMid(pSettings_, pCommand, iPipe1, iPipe2);
                }
            }
            _exit(0);
        }
    } while(pCommand != NULL);

    return;
}

Solution

  • It is my assumption that the forks should terminate and return back to the parent process after the execvp() is done

    If you mean this the way I think you do, then you're wrong. The exec family of functions do not destroy the child process. They swap out the program executing in the child process for a different one, but the process remains the same. In particular, waitpid for that process does not return until the program invoked by exec terminates.

    This would be fine if you were just creating one child process which would run to completion without further help from the parent. But you're trying to set up a pipeline. You can't expect any of the processes in a pipeline to terminate until they are all finished. You especially can't expect the first process in a pipeline to terminate before the second one has even been created. But that is exactly what your code does: it waits for the first process to terminate before it creates the second one -- meanwhile, the first process has filled up its pipe and is waiting for the second process to read some of the data before it can go on. Deadlock.

    What you need to do instead, is create all of the processes before waiting for any of them. You've already got one of the two loops you need. Instead of calling waitpid where you do, save the process ID in a data structure and go to the next iteration. Then, in a second loop after you've consumed the entire "command list", call waitpid in a loop without specifying any particular process ID to wait for; as each process terminates, remove its ID from the data structure where you saved them; when that data structure is empty, you're done.

    Also, you need to check each and every system call for errors, except close. (It is a bug in POSIX that close is allowed to fail; sensible implementations of Unix only return an error if passed a file descriptor number that wasn't open in the first place, and I have yet to encounter a program that needed to care about that. If you need to ensure data has hit the disk, you need to call fsync before close; but doing an fsync on a pipe is pointless.)

    While I'm here, some observations on good programming style: (1) your code will be easier to read if you stop using Hungarian prefixes. (1a) meaningless underscores at the end of identifiers are reserved by convention for variables defined inside macros, C not having macro hygiene. (1b) function names that begin with underscores are reserved by the C standard for the C library's internal use; your wrapper around fork should be named forkWithSettings -- something even more descriptive would be better, but not knowing what-all is in this settings object I can't suggest anything. (2) it doesn't matter all that much in the grand scheme of things where you put your curly braces, but for the love of Ghu, pick one style and use it consistently throughout the entire file.