How to test/validate the vmalloc guard page is working in Linux

I am studying stack guarding in Linux. I found that the Linux kernel VMAP_STACK config parameter is using the guard page mechanism along with vmalloc() to provide stack guarding.
I am trying to find a way to check how this guard page is working in Linux kernel. I googled and checked the kernel code, but did NOT find out the codes.

A further question is how to verify the guarded stack.
I had a kernel module to underrun/overflow a process's kernel stack, like this

static void shoot_kernel_stack(void)
{
    unsigned char *ptr =  task_stack_page(current);
    unsigned char *tmp = NULL;


    tmp = ptr + THREAD_SIZE + PAGE_SIZE + 0;
//  tmp -= 0x100;
    memset(tmp, 0xB4, 0x10); // Underrun
}

I really get the kernel panic like below,

[ 8006.358354] BUG: stack guard page was hit at 00000000e8dc2d98 (stack is 00000000cff0f921..00000000653b24a9)
[ 8006.361276] kernel stack overflow (page fault): 0000 [#1] SMP PTI

Is this the right way to verify the guard page?

Solution

The VMAP_STACK Linux feature is used to map the kernel stack of the threads into VMA. By virtually mapping stack, the underlying physical pages don't need to be contiguous. It is possible to detect cross-page overflows by adding guard pages. As the VMA are followed by a guard (unless the VM_NO_GUARD flag is passed at allocation time), the stacks allocated in those area benefits from it for stack overflow detection.

ALLOCATION

The thread stacks are allocated at thread creation time with alloc_thread_stack_node() in kernel/fork.c. When VMAP_STACK is activated, the stacks are cached because according to the comments in the source code:

vmalloc() is a bit slow, and calling vfree() enough times will force a TLB flush. Try to minimize the number of calls by caching stacks.

The kernel stack size is THREAD_SIZE (equal to 4 pages on x86_64 platforms). The source code of the allocation invoked at thread creation time is:

static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node)
{
#ifdef CONFIG_VMAP_STACK
    void *stack;
    int i;

[...] // <----- Part which gets a previously cached stack. If no stack in cache
      // the following is run to allocate a brand new stack:

    /*
     * Allocated stacks are cached and later reused by new threads,
     * so memcg accounting is performed manually on assigning/releasing
     * stacks to tasks. Drop __GFP_ACCOUNT.
     */
    stack = __vmalloc_node_range(THREAD_SIZE, THREAD_ALIGN,
                     VMALLOC_START, VMALLOC_END,
                     THREADINFO_GFP & ~__GFP_ACCOUNT,
                     PAGE_KERNEL,
                     0, node, __builtin_return_address(0));
[...]

__vmalloc_node_range() is defined in mm/vmalloc.c. This calls __get_vm_area_node(). As the latter is not passed the VM_NO_GUARD flags, an additional page is added at the end of the allocated area. This is the guard page of the VMA:

static struct vm_struct *__get_vm_area_node(unsigned long size,
        unsigned long align, unsigned long flags, unsigned long start,
        unsigned long end, int node, gfp_t gfp_mask, const void *caller)
{
    struct vmap_area *va;
    struct vm_struct *area;

    BUG_ON(in_interrupt());
    size = PAGE_ALIGN(size);
    if (unlikely(!size))
        return NULL;

    if (flags & VM_IOREMAP)
        align = 1ul << clamp_t(int, get_count_order_long(size),
                       PAGE_SHIFT, IOREMAP_MAX_ORDER);

    area = kzalloc_node(sizeof(*area), gfp_mask & GFP_RECLAIM_MASK, node);
    if (unlikely(!area))
        return NULL;

    if (!(flags & VM_NO_GUARD)) // <----- A GUARD PAGE IS ADDED
        size += PAGE_SIZE;

    va = alloc_vmap_area(size, align, start, end, node, gfp_mask);
    if (IS_ERR(va)) {
        kfree(area);
        return NULL;
    }

    setup_vmalloc_vm(area, va, flags, caller);

    return area;
}

OVERFLOW MANAGEMENT

The stack overflow management is architecture dependent (i.e. source code located in arch/...). The links referenced below provide some pointers on some architecture dependent implementations.

For x86_64 platform, the overflow check is done upon the page fault interruption which triggers the following chain of function calls: do_page_fault()->__do_page_fault()->do_kern_addr_fault()->bad_area_nosemaphore()->no_context() function defined in arch/x86/mm/fault.c. In no_context(), there is a part dedicated to VMAP_STACK management for the detection of the stack under/overflow:

static noinline void
no_context(struct pt_regs *regs, unsigned long error_code,
       unsigned long address, int signal, int si_code)
{
    struct task_struct *tsk = current;
    unsigned long flags;
    int sig;
[...]
#ifdef CONFIG_VMAP_STACK
    /*
     * Stack overflow?  During boot, we can fault near the initial
     * stack in the direct map, but that's not an overflow -- check
     * that we're in vmalloc space to avoid this.
     */
    if (is_vmalloc_addr((void *)address) &&
        (((unsigned long)tsk->stack - 1 - address < PAGE_SIZE) ||
         address - ((unsigned long)tsk->stack + THREAD_SIZE) < PAGE_SIZE)) {
        unsigned long stack = __this_cpu_ist_top_va(DF) - sizeof(void *);
        /*
         * We're likely to be running with very little stack space
         * left.  It's plausible that we'd hit this condition but
         * double-fault even before we get this far, in which case
         * we're fine: the double-fault handler will deal with it.
         *
         * We don't want to make it all the way into the oops code
         * and then double-fault, though, because we're likely to
         * break the console driver and lose most of the stack dump.
         */
        asm volatile ("movq %[stack], %%rsp\n\t"
                  "call handle_stack_overflow\n\t"
                  "1: jmp 1b"
                  : ASM_CALL_CONSTRAINT
                  : "D" ("kernel stack overflow (page fault)"),
                "S" (regs), "d" (address),
                [stack] "rm" (stack));
        unreachable();
    }
#endif
[...]
}

In the above code, when a stack under/overflow is detected, the handle_stack_overflow() function defined in arch/x86/kernel/traps.c) is called:

#ifdef CONFIG_VMAP_STACK
__visible void __noreturn handle_stack_overflow(const char *message,
                        struct pt_regs *regs,
                        unsigned long fault_address)
{
    printk(KERN_EMERG "BUG: stack guard page was hit at %p (stack is %p..%p)\n",
         (void *)fault_address, current->stack,
         (char *)current->stack + THREAD_SIZE - 1);
    die(message, regs, 0);

    /* Be absolutely certain we don't return. */
    panic("%s", message);
}
#endif

The example error message "BUG: stack guard page was hit at..." pointed out in the question comes from the above handle_stack_overflow() function.

FROM YOUR EXAMPLE MODULE

When VMAP_STACK is defined, the stack_vm_area field of the task descriptor appears and is set with the VMA address associated to the stack. From there, it is possible to grab interesting information:

struct task_struct *task;
#ifdef CONFIG_VMAP_STACK
struct vm_struct *vm;
#endif // CONFIG_VMAP_STACK

task = current;

printk("\tKernel stack: 0x%lx\n", (unsigned long)(task->stack));
printk("\tStack end magic: 0x%lx\n", *(unsigned long *)(task->stack));
#ifdef CONFIG_VMAP_STACK
vm = task->stack_vm_area;
printk("\tstack_vm_area->addr = 0x%lx\n", (unsigned long)(vm->addr));
printk("\tstack_vm_area->nr_pages = %u\n", vm->nr_pages);
printk("\tstack_vm_area->size = %lu\n", vm->size);
#endif // CONFIG_VMAP_STACK
printk("\tLocal var in stack: 0x%lx\n", (unsigned long)(&task));

The nr_pages field is the number of pages without the additional guard page. The last unsigned long at the top of the stack is set with STACK_END_MAGIC defined in include/uapi/linux/magic.h as:

#define STACK_END_MAGIC     0x57AC6E9D

REFERENCES: