gracefully shutdown (multi-threaded) gSOAP service with http-keepalive enabled

Question

I have a multi-threaded gSOAP service running with enabled http-keepalive. How can I gracefully shutdown the service when there are still clients connected?

A similar question was asked in gSoap: how to gracefully shutdown the webservice application?, but the answers do not cover the http-keepalive aspect: The soap-serve function will simply not return until the http-keepalive-session wasn't closed by the client. Thus, step 2 in the accepted answer will block until the client decides to close the connection (or the receive-timeout expires, but a short timeout would break the desired http-keepalive behaviour here).

The examples from the gSOAP documentation suffer from the same problem.

What I tried so far was to call soap_done() for all soap structs that are hanging in a soap_serve call from the main thread to interrupt the connections waiting for http-keepalive, which works most of the time, but crashes in rare conditions (a race condition maybe), so this is no solution for me.

Answer 1

I just ran into the very same problem and I think I've got a solution for you.

As you just said, the problem is that the gSoap hangs on soap_serve. This happens because gSOAP generates an internal loop for you that waits for the arrival of all keep-alive requests OR a timeout on the server-side arises.

What I've done is grabbing the soap_serve function inside the automatically generated service stub. I'm going to list the original soap_serve function so that you can find it on your service stub file :

  SOAP_FMAC5 int SOAP_FMAC6 soap_serve(struct soap *soap)
    {
    #ifndef WITH_FASTCGI
        unsigned int k = soap->max_keep_alive;
    #endif

        do
        {
    #ifdef WITH_FASTCGI
            if (FCGI_Accept() < 0)
            {
                soap->error = SOAP_EOF;
                return soap_send_fault(soap);
            }
    #endif

            soap_begin(soap);

    #ifndef WITH_FASTCGI
            if (soap->max_keep_alive > 0 && !--k)
                soap->keep_alive = 0;
    #endif

            if (soap_begin_recv(soap))
            {   if (soap->error < SOAP_STOP)
                {
    #ifdef WITH_FASTCGI
                    soap_send_fault(soap);
    #else 
                    return soap_send_fault(soap);
    #endif
                }
                soap_closesock(soap);

                continue;
            }

            if (soap_envelope_begin_in(soap)
             || soap_recv_header(soap)
             || soap_body_begin_in(soap)
             || soap_serve_request(soap)
             || (soap->fserveloop && soap->fserveloop(soap)))
            {
    #ifdef WITH_FASTCGI
                soap_send_fault(soap);
    #else
                return soap_send_fault(soap);
    #endif
            }

    #ifdef WITH_FASTCGI
            soap_destroy(soap);
            soap_end(soap);
        } while (1);
    #else
        } while (soap->keep_alive);
    #endif
        return SOAP_OK;
    }

You should extract the body of this function and replace your old soap_serve(mySoap) call inside your thread (the thread that performs the requests and hagns because of the keep-alive) with the following:

do
    {
        if ( Server::mustShutdown() ) {
            break;
        }

        soap_begin(mySoap);

        // If we reached the max_keep_alive we'll exit
        if (mySoap->max_keep_alive > 0 && !--k)
            mySoap->keep_alive = 0;


        if (soap_begin_recv(mySoap))
        {   if (mySoap->error < SOAP_STOP)
            {
                soap_send_fault(mySoap);
                break;
            }
            soap_closesock(mySoap);

            continue;
        }

        if (soap_envelope_begin_in(mySoap)
         || soap_recv_header(mySoap)
         || soap_body_begin_in(mySoap)
         || soap_serve_request(mySoap)
         || (mySoap->fserveloop && mParm_Soap->fserveloop(mySoap)))
        {
            soap_send_fault(mySoap);
            break;
        }


    } while (mySoap->keep_alive);

Note the following:

1. The Server::mustShutdown() acts as a flag that will be set to true (externally) to end all the threads. When you want to stop the server from handling new requests you'll this function will return true and the loop will end.
2. I've removed the ifdef, WITH_FASTCGI it's not interesting for us now.
3. When the you close the connection like this, any clients connected to the server will raise an exception. Clients written in C# for instance will throw a "The underlying connection is excepted to keep alive was closed by the server" wich makes perfect sense for us.

But we are not done yet, thanks to what AudioComplex pointed out, the system still remains waiting for reqeuests on soap_begin_recv. But I've got a solution for that too ;)

Each of the threads on the connection-handling pool creates a copy of the main soap context (via soap_copy), these threads are the ones that
I store each of these contexts as an element on the array that resides on the main connection-handling thread. When terminating the main connection-handling thread (the one that serves the requests) it will go through all soap contexts and finalize "manually" the connection by using:

for (int i = 0; i < soaps.size(); ++i) {
  soaps[i]->fclose(soaps[i]);
}

This will force the soap_serve loop to finish. It actually will stop the internal loop near line 921 of stdsoap2.cpp_

r = select((int)soap->socket + 1, &fd, NULL, &fd, &timeout);

It is not the cleanest solution (haven't found a cleaner one) but it will definitely stop the service.