How does a thread in NPTL exit?

Question

I'm curious how a single NPTL thread exits, from implementation perspective.

What I understand about glibc-2.30's implementation are:

1. NPTL thread is built on top of light weight process on Linux, with additional information stored in pthread object on user stack, to keep track of NPTL specific information such as join/detach status and returned object's pointer.
2. when a NPTL thread is finished, it is gone for good, only the user stack (and hence) pthread object is left to be collected (to be joined by other threads), unless it is a detached, in which case that space is directly freed.
3. _exit() syscall kills all threads in a thread group.
4. the user function that pthread_create() takes in is actually wrapped into another function start_thread(), which does some preparation before running the user function, and some cleaning up afterwards.

Questions are:

1. At the end of the wrapper function start_thread(), there are the following comment and code:

   /* We cannot call '_exit' here.  '_exit' will terminate the process.
   
    The 'exit' implementation in the kernel will signal when the
    process is really dead since 'clone' got passed the CLONE_CHILD_CLEARTID
    flag.  The 'tid' field in the TCB will be set to zero.
   
    The exit code is zero since in case all threads exit by calling
    'pthread_exit' the exit status must be 0 (zero).  */
    __exit_thread ();
   

   but __exit_thread() seems to do syscall _exit() anyways:

    static inline void __attribute__ ((noreturn, always_inline, unused))
    __exit_thread (void)
    {
      /* some comments here */
      while (1)
        {
          INTERNAL_SYSCALL_DECL (err);
          INTERNAL_SYSCALL (exit, err, 1, 0);
        }
    }
   

   so I'm confused here, since it shouldn't really do syscall _exit() because it will terminate all threads.

2. pthread_exit() should terminate a single thread, so it should do something similar to what the wrapper start_thread() does in the end, however it calls __do_cancel(), and TBH I am lost in tracing down that function. It does not seem to be related to the above __exit_thread(), nor does it call _exit().

Answer 1

I'm confused here, since it shouldn't really do syscall _exit()

The confusion here stems from mixing exit system call with _exit libc routine (there is no _exit system call on Linux).

The former terminates current Linux thread (as intended).

The latter (confusingly) doesn't execute exit system call. Rather, it executes exit_group system call, which terminates all threads.

thread_exit() should terminate a single thread

It does, indirectly. It unwinds current stack (similar to siglongjmp), performing control transfer to the point where cleanup_jmp_buf was set up. And that was in start_thread.

After the control transfer, start_thread cleans up resources, and calls __exit_thread to actually terminate the thread.