Why is this C function pointer defined in the stack not valid when passed?

Question

I am attempting to pass a function pointer defined within the parent block scope to another function. I get both working and segfaults in different environments. (I am not a C expert)

The code:

#include <stdio.h>
#include <stdbool.h>

void test_function(bool (*function_pointer) (int x)) {
        printf("addr passed function_pointer %p\n", function_pointer);
        if (function_pointer(100)) {
                printf("  run: true\n");
        } else {
                printf("  run: false\n");
        }
}

bool function_outside_main(int x) {
        return x < 0;
}

int main(void) {
        // run with function defined globally
        printf("addr function_outside_main %p\n", function_outside_main);
        test_function(function_outside_main);

        // run with function defined in this stack block
        bool function_inside_main(int x) {
                return x > 0;
        }
        printf("addr function_inside_main %p\n", function_inside_main);
        test_function(function_inside_main); // shouldn't the address be valid?
}

On Ubuntu 16.04.4 with GCC version 5.4.0 (on an Amazon EC2) it works with output:

addr function_outside_main 0x400620
addr passed function_pointer 0x400620
  run: false
addr function_inside_main 0x7ffc018d5690
addr passed function_pointer 0x7ffc018d5690
  run: true

On Ubuntu 20.04.1 with GCC version 9.3.0 (under Windows WSL) it fails with a segfault:

addr function_outside_main 0x7ff19c8631dd
addr passed function_pointer 0x7ff19c8631dd
  run: false
addr function_inside_main 0x7ffffc033b20
addr passed function_pointer 0x7ffffc033b20
zsh: segmentation fault (core dumped)  ./a.out

Answer 1

Nested functions like this is a gcc extension, not part of the C standard.

The implementation used by gcc for this generally¹ involves creating an on-stack thunk for the nested function, so calling it requires executable stack support. More recent versions of Linux (and Windows) default to a non-executable stack, so will crash.

To make this work, you can use the -z execstack option to gcc, or you can use the execstack tool to modify the binary to specify an executable stack after creating it.

---

¹_{In some versions of gcc with -O it can determine when nested functions don't actually need to be nested (they never reference the containing scope), and not use the thunk for those cases}