Search code examples
clinuxtimerblocking

How to execute code post blocking operation after expiry of timer in C/Linux?

I have a situation in which I need to start an expiry timer before a blocking operation. In case control doesn't come out of the blocking operation within a predetermined time interval, I need to go ahead further and resume other activities. The program should not be terminated upon the expiry of the timer. Rather I have to execute code that comes subsequent to the blocking operation. In the following snippet, upon expiry of timer, I need to go to point B and not exit. Working environment is ubuntu 18.04 and the code is in c. I have explored setitimer function call along with associated with signal handler. But control is not skipping the blocking operation and going ahead to subsequent code.

  Program to create a semaphore and setting the semaphore for use by another program.

  int main(int argc,char *argv[])
  {
    key_t SemKey = 0x12345 ;

    int SemId = -1;
    int NumOfSems = 2 ;
    int Flag ;
    int RetStat ;

    char choice = 'n' ;

    struct
    {
            int val;
            struct semid_ds *buf;
            ushort array[2] ;
    } Arg ;


    Flag = IPC_CREAT | IPC_EXCL | 0666 ;

    SemId = semget(SemKey,NumOfSems,Flag ) ;
    if ( SemId == -1 )
    {
            perror("semget");
            exit(EXIT_FAILURE);
    }

    Arg.array[0] = 0 ;
    Arg.array[1] = 0 ;

    RetStat = semctl(SemId, 0, SETALL, Arg.array);

    if (RetStat == -1)
    {
            perror("semctl");
            exit(EXIT_FAILURE);
    }

    printf("\nPress y or Y to continue ahead.\n");
    scanf("%c",&choice);

    if ( ( choice != 'y') && ( choice != 'Y'))
            exit(EXIT_FAILURE);

    if ( ( choice != 'y') && ( choice != 'Y'))
            exit(EXIT_FAILURE);


    RetStat = semctl( SemId,0, SETVAL, 1 );
    if( RetStat < 0 )
    {
            perror( "SET SEMOP Failure: " );
            exit(EXIT_FAILURE);
    }

    return EXIT_SUCCESS ;
  }

  Program that waits for the semaphore set by the above program.

  int DriverModule(int );

  int main(int argc,char *argv[])
  {
    key_t SemKey = 0x12345 ;
    int SemId ;
    u_int Flag;

    Flag = 0666 ;

    SemId = semget( SemKey, 0, Flag );
    if ( SemId == -1 )
    {
            perror("semget");
            exit(EXIT_FAILURE);
    }

    DriverModule(SemId) ;

    return EXIT_SUCCESS ;
  }

  #define MAX_ITERATIONS 100

  struct itimerval timer;

  int WaitForSemaphore(int ,unsigned short ) ;

  void timer_handler (int signum)
  {
    static int count = 0;
    printf ("timer expired %d times\n", ++count);

    if ( count >= MAX_ITERATIONS ) // Stop the timer
    {
            printf("\n\nForcing Time Out Termination.....\n\n");

            timer.it_value.tv_sec = 0;
            timer.it_value.tv_usec = 0;
            timer.it_interval.tv_sec = 0;
            timer.it_interval.tv_usec = 0;

            setitimer (ITIMER_VIRTUAL, &timer, NULL);

            return ;
    }

   }

  int DriverModule (int SemId)
  {
    struct sigaction sa;

    /* Install timer_handler as the signal handler for SIGVTALRM. */
    memset (&sa, 0, sizeof (sa));

    sa.sa_flags = SA_SIGINFO;
    sa.sa_handler = &timer_handler;
    sigaction (SIGVTALRM, &sa, NULL);

    /* Configure the timer to expire after 250 msec... */
    timer.it_value.tv_sec = 0;
    timer.it_value.tv_usec = 250000;
    /* ... and every 500 msec after that. */
    timer.it_interval.tv_sec = 0;
    timer.it_interval.tv_usec = 500000;

    /* Start a virtual timer. It counts down whenever this process is  
     executing. */
    setitimer (ITIMER_VIRTUAL, &timer, NULL);

    printf("\nBefore calling wait for semaphore.......\n");

    // Waiting for sempahore
    if( !WaitForSemaphore( SemId, 0) )
    {
            printf("\nUnable to get sempahore.\n");
            return 0 ;
    }

    printf("\nAfter calling after calling wait for semaphoe module.........\n");

    return 1 ;
  }

  int WaitForSemaphore(int SemId,unsigned short SemNum )
  {
    struct sembuf SemBuf;

    int  RetStat;
    unsigned int NoOfSemOps;

    SemBuf.sem_num = SemNum;
    SemBuf.sem_op = -1;
    SemBuf.sem_flg = 0;

    NoOfSemOps = 1;

    RetStat = semop( SemId, &SemBuf, NoOfSemOps );
    if( RetStat < 0 )
    {
            if( errno == EINTR )
            {
                    WaitForSemaphore( SemId, SemNum );
            }
            else
            {
                    perror( "Wait SEMOP Failure: " );
                    return 0 ;
            }
    }

    return 1 ;

   }
Solution

  • Okay, the main issue is using ITIMER_VIRTUAL. This [type of] counter only decrements when a process is running. A process is not running if it is performing a syscall.

    So, we have to use ITIMER_REAL instead. And, if we do that it generates a SIGALRM signal [and not a SIGVTALRM one].

    After making those changes, there is another issue.

    The timer must be disabled after the syscall it is protecting (e.g. the semop). Otherwise, an unexpired timer (i.e. the semop did not timeout) could interrupt another/subsequent/unrelated syscall (e.g. read, write, etc.).

    So, in the code below, we need (e.g.):

    timer_set(250000);
semop(...);
timer_set(0);

    Also, note that there a limited number of syscalls [and/or library function calls] that can be performed [safely] from a signal handler. Notably, printf can not be used in a signal handler.

    And, even doing a "signal safe" syscall would require that the signal handler save [upon entry] and restore [upon exit] the original value for errno.

    Otherwise, the baseline code (e.g. after the [interrupted] semop) would not see the correct errno value (e.g. EINTR) but the errno value for whatever syscall(s) the signal handler chose to do.

    I had to refactor the code a fair amount to make a reasonable test program.

    I combined both programs into a single test program in order to generate a proper unit test for both the normal and timeout cases, which would have been difficult with them as separate programs due to timing issues and race conditions.

    I also enhanced the debug printout.

    Anyway, here is the code:

    // create a semaphore and setting the semaphore for use by another program.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <signal.h>
#include <unistd.h>
#include <stdarg.h>
#include <time.h>

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
#include <sys/time.h>
#include <sys/wait.h>

int opt_n;
int opt_y;

#define MAX_ITERATIONS 100

int itype;
struct itimerval timer;
const char *pgmtail;

int WaitForSemaphore(int, unsigned short);
int DriverModule(int);

typedef long long tsc_t;

tsc_t tvzero;

tsc_t
tscget(void)
{
    struct timespec ts;
    tsc_t tsc;

    clock_gettime(CLOCK_MONOTONIC,&ts);

    tsc = ts.tv_sec;
    tsc *= 1000000000;
    tsc += ts.tv_nsec;

    tsc -= tvzero;

    return tsc;
}

double
tscsec(tsc_t tsc)
{
    double sec;

    sec = tsc;
    sec /= 1e9;

    return sec;
}

void
dbgprt(const char *fmt,...)
{
    va_list ap;
    char buf[1000];
    char *bp = buf;

    bp += sprintf(bp,"%.9f %s ",tscsec(tscget()),pgmtail);

    va_start(ap,fmt);
    bp += vsprintf(bp,fmt,ap);
    va_end(ap);

    fputs(buf,stdout);
    fflush(stdout);
}

int
pgma(void)
{
    key_t SemKey = 0x12345;

    pgmtail = "pgma";

    int SemId = -1;
    int NumOfSems = 2;
    int Flag = 0;
    int RetStat;

#if 0
    char choice = 'n';
#endif

#if 0
    struct {
        int val;
        struct semid_ds *buf;
        ushort array[2];
    } Arg;
#endif

    Flag |= IPC_CREAT;
    //Flag |= IPC_EXCL;
    Flag |= 0666;

    SemId = semget(SemKey, NumOfSems, Flag);
    if (SemId == -1) {
        perror("semget");
        exit(EXIT_FAILURE);
    }

#if 0
    Arg.array[0] = 0;
    Arg.array[1] = 0;

    RetStat = semctl(SemId, 0, SETALL, Arg.array);

    if (RetStat == -1) {
        perror("semctl");
        exit(EXIT_FAILURE);
    }
#endif

    for (int phase = 0;  phase <= opt_y;  ++phase) {
        int setval = phase;
        dbgprt("pgma: SET phase=%d\n",phase);

        RetStat = semctl(SemId, 0, SETVAL, setval);
        if (RetStat < 0) {
            perror("SET SEMOP Failure: ");
            exit(EXIT_FAILURE);
        }

        dbgprt("USLEEP/BEF\n");
        usleep(1000 * 1000);
        dbgprt("USLEEP/AFT\n");
    }

    return EXIT_SUCCESS;
}

// Program that waits for the semaphore set by the above program.

int
pgmb(void)
{
    key_t SemKey = 0x12345;
    int SemId;

    pgmtail = "pgmb";

// NOTE/BUG: we must add IPC_CREAT if we start pgmb first
    u_int Flag = 0;
    Flag |= 0666;
#if 1
    Flag |= IPC_CREAT;
#endif
    SemId = semget(SemKey, 2, Flag);

    if (SemId == -1) {
        perror("semget");
        exit(EXIT_FAILURE);
    }

    DriverModule(SemId);

    dbgprt("pgmb: complete\n");

    return EXIT_SUCCESS;
}

char *
exmsg(int status)
{
    static char buf[1000];
    char *bp = buf;

    bp += sprintf(bp,"status=[%8.8X ",status);

    do {
        if (WIFEXITED(status)) {
            bp += sprintf(bp,"exited status=%d", WEXITSTATUS(status));
            break;
        }

        if (WIFSIGNALED(status)) {
            bp += sprintf(bp,"killed by signal %d", WTERMSIG(status));
            break;
        }

        if (WIFSTOPPED(status)) {
            bp += sprintf(bp,"stopped by signal %d", WSTOPSIG(status));
            break;
        }

        bp += sprintf(bp,"continued");
    } while (0);

    bp += sprintf(bp,"]");

    return buf;
}

void
testit(int newval)
{
    int code;
    pid_t pid;

    opt_y = newval;

    for (int phase = 1;  phase <= 2;  ++phase) {
        // start the receiver
        pid = fork();
        if (pid != 0) {
            dbgprt("testit: pid=%d\n",pid);
            continue;
        }

        switch (phase) {
        case 1:
            code = pgma();
            break;
        case 2:
            code = pgmb();
            break;
        default:
            code = 99;
            break;
        }

        exit(code);
    }

    while (1) {
        int status;

        pid = wait(&status);
        if (pid <= 0)
            break;

        dbgprt("main: pid %d reaped -- %s\n",pid,exmsg(status));
    }
}

int
main(int argc,char **argv)
{
    //pid_t pid;

    --argc;
    ++argv;

    pgmtail = "main";

    tvzero = tscget();

    for (;  argc > 0;  --argc, ++argv) {
        char *cp = *argv;
        if (*cp != '-')
            break;

        switch (cp[1]) {
        case 'n':
            opt_n = ! opt_n;
            break;
        }
    }

    opt_y = ! opt_n;
    dbgprt("pgma: opt_y=%d\n",opt_y);

    //testit(0);
    testit(1);

    return 0;
}

void
timer_set(int usec)
{

    dbgprt("timer_set: SET usec=%d\n",usec);

    timer.it_value.tv_sec = 0;
    timer.it_value.tv_usec = usec;

    /* ... and every 500 msec after that. */
// NOTE/BUG: timer must _not_ be rearmed -- can abort unrelated syscalls
    timer.it_interval.tv_sec = 0;
#if 0
    timer.it_interval.tv_usec = 500000;
#else
    timer.it_interval.tv_usec = 0;
#endif

    setitimer(itype, &timer, NULL);
}

void
timer_handler(int signum)
{
    static int count = 0;

// NOTE/BUG: printf may _not_ be called from a signal handler
    //dbgprt("timer expired %d times\n", ++count);

    // Stop the timer
// NOTE/BUG: disabling now done elsewhere
    if (count >= MAX_ITERATIONS) {
        //dbgprt("Forcing Time Out Termination.....\n");

        timer.it_value.tv_sec = 0;
        timer.it_value.tv_usec = 0;
        timer.it_interval.tv_sec = 0;
        timer.it_interval.tv_usec = 0;

#if 0
        setitimer(ITIMER_VIRTUAL, &timer, NULL);
#else
        setitimer(ITIMER_REAL, &timer, NULL);
#endif

        return;
    }
}

int
DriverModule(int SemId)
{
    struct sigaction sa;
    sigset_t set;
    int signo;

#if 0
    itype = ITIMER_VIRTUAL;
    signo = SIGVTALRM;
#else
    itype = ITIMER_REAL;
    signo = SIGALRM;
#endif

    /* Install timer_handler as the signal handler for SIGVTALRM. */
    memset(&sa, 0, sizeof(sa));

    sa.sa_flags = SA_SIGINFO;
    sa.sa_handler = &timer_handler;
    sigaction(signo, &sa, NULL);

    sigemptyset(&set);
    sigaddset(&set,signo);
    sigprocmask(SIG_UNBLOCK,&set,NULL);

    /* Configure the timer to expire after 250 msec... */

    // Start virtual timer. It counts down whenever this process is executing.
#if 0
    setitimer(itype, &timer, NULL);
#else
    timer_set(250000);
#endif

    dbgprt("Before calling wait for semaphore.......\n");

    // Waiting for sempahore
    if (! WaitForSemaphore(SemId, 0)) {
        dbgprt("Unable to get sempahore.\n");
        return 0;
    }

    dbgprt("After calling after calling wait for semaphoe module.........\n");

    return 1;
}

int
WaitForSemaphore(int SemId, unsigned short SemNum)
{
    struct sembuf SemBuf;

    int RetStat;
    unsigned int NoOfSemOps;

    while (1) {
        SemBuf.sem_num = SemNum;
        SemBuf.sem_op = -1;
        SemBuf.sem_flg = 0;

        NoOfSemOps = 1;

        dbgprt("WaitFor: SEMOP\n");

        RetStat = semop(SemId, &SemBuf, NoOfSemOps);
        if (RetStat >= 0) {
            dbgprt("WaitFor: OKAY\n");
            break;
        }

        if (errno == EINTR) {
            dbgprt("WaitFor: EINTR\n");

            // do stuff here ...

            // rearm timer
            timer_set(250000);
            continue;
        }

        perror("Wait SEMOP Failure: ");
        exit(1);
    }

// NOTE/BUG: _must_ always disable timer to prevent other syscalls from being
// interrupted
#if 1
    timer_set(0);
#endif

    return 1;
}

    Here is some sample output from the refactored program:

    0.000000332 main pgma: opt_y=1
0.000182324 main testit: pid=849558
0.000267203 main testit: pid=849559
0.000830005 pgma pgma: SET phase=0
0.000847541 pgmb timer_set: SET usec=250000
0.000882037 pgma USLEEP/BEF
0.000891077 pgmb Before calling wait for semaphore.......
0.000895977 pgmb WaitFor: SEMOP
0.250932859 pgmb WaitFor: EINTR
0.250950128 pgmb timer_set: SET usec=250000
0.250956676 pgmb WaitFor: SEMOP
0.500996272 pgmb WaitFor: EINTR
0.501014687 pgmb timer_set: SET usec=250000
0.501021903 pgmb WaitFor: SEMOP
0.751066428 pgmb WaitFor: EINTR
0.751089504 pgmb timer_set: SET usec=250000
0.751097693 pgmb WaitFor: SEMOP
1.000970921 pgma USLEEP/AFT
1.000987303 pgma pgma: SET phase=1
1.001001916 pgma USLEEP/BEF
1.001046071 pgmb WaitFor: OKAY
1.001055982 pgmb timer_set: SET usec=0
1.001062505 pgmb After calling after calling wait for semaphoe module.........
1.001066632 pgmb pgmb: complete
1.001210687 main main: pid 849559 reaped -- status=[00000000 exited status=0]
2.001078396 pgma USLEEP/AFT
2.001269995 main main: pid 849558 reaped -- status=[00000000 exited status=0]

    UPDATE:

    1. Why re-arming the timer is required inside WaitForSemaphore?

    By commenting out the printf calls in the signal handler [as we must do because of the signal safety issue], as a side effect, the increment of count is nullified.

    At first, I just commented out the printf, not realizing that the side effect "broke" the handler's disarm code. After a while, I realized what I had done and wanted the effect (i.e. have the handler "do nothing").

    Thus, the entire signal handler [effectively] does nothing except to allow the signal to be intercepted [and cause a pending syscall to receive an EINTR error--which is what we really want].

    I neglected to explicitly state this in a comment there [and comment out all code with an #if 0].

    As an example, because of my initial confusion, it's probably better to not put active/necessary code [the count++] inside debug code (the printf). So, printf(...,count++); would be better as: printf(...); count++;

    This was an architectural choice to have the baseline code [the non-handler code], control the arm/disarm. This is because the baseline does not know if the handler did anything, so it has to do the disarm explicitly anyway [to prevent the timer/signal from (mis)firing outside of the wait loop code].

    In general, having a periodic timer interrupt [in a kernel] is certainly useful, and by analogy, somewhat useful as a periodic signal, this isn't as helpful for the use case here of just waking up a [stuck] syscall.

    1. Why usleep is required inside pgma module?

    This is just to force pgmb to see both conditions of a timed out call to semop and a normal one for testing/validation purposes. In normal/production code, there would be no need for pgma to sleep.

    At its start, pgma posts a 0 value to the semaphore. It then sleeps for 1 sec (1000 msec).

    When pgmb starts, it will see this 0 value [and timeout after ~250 msec]. pgmb will loop on this approximately 4 times, until pgma wakes up and posts a 1 value [which will complete pgmb's semop normally].

    So, now, we've gotten pgmb to experience both the failed/timeout case and the normal/successful case.

    This is what a [good] set of unit tests is supposed to do: test that the target code [the wait loop for semop in pgmb] is correct for all possible modes/states.

    In this code, we see that by looking at the debug output. For a true set of unit tests, we'd need additional code that programatically checked for this.