Search code examples
assemblylocal-variablescalling-conventionstack-memorynested-function

Could a callee directly access a stack variable in the caller without pointers in theory?


C is pass-by-value which means that all the parameters are copied to the stack frame every time you call a function. C also does not support inner functions that could access/modify local variables in the lexically enclosing function, although GNU C does as an extension. The only way to 'pass by reference' in C is to pass a pointer. While there are reference types in C++ these are just pointers behind the scenes.

I thought about the following scenario in a hypothetical low-level C-like language. There could be a function declared inside another function only visible to the outer function and will only be called by the outer function. If my understanding is correct the stack pointer is incremented by a fixed amount when the inner function is called assuming no funny business such as variable-sized objects. Therefore as the thought process goes any local variables declared in the outer function would be at fixed offsets relative to the stack pointer even inside the inner function. They could in theory be accessed in the same way as other local variables without needing to pass a pointer. And of course trying to access and call the inner function from outside the the outer function would try to access out-of-lifetime local variables in the outer function and result in undefined behavior.

Here is an illustration: (again this not C but an imaginary C-like language)

void (*address_to_inner)(void);
void outer(void) {
    int a = 10;
    void inner(void) {
        printf("%d\n", a);
    }
    inner(); // Prints 10
    address_to_inner = inner; // Not undefined behavior yet but not good
}
void another(void) {
    outer(); // Prints 10
    address_to_inner(); // Undefined behavior because inner() tries to access `a` after it was deallocated
}

Does my thought process apply on typical architectures such as x86 or ARM? Some features such as automatic memory management are not supported in low-level languages such as C because they are not natively supported by architectures and would require too much complex behind-the-scenes work to be feasible for a low-level language. But is my scenario 'feasible' for a low-level language or are there any architectural or technical limitations that I did not consider that would make implementing this non-trivial based on how the stack/local variables and function calls work? If not why not?


Solution

  • Therefore as the thought process goes any local variables declared in the outer function would be at fixed offsets relative to the stack pointer even inside the inner function.

    This would work, though you would have to preclude function calling among nested function, which also precludes recursion among them.  Let's note that Pascal nested functions can call each other, and recursively as well.

    Pascal's stack-based static link mechanism was appropriate for a time when machines had few CPU registers, so most any Pascal program variables were mapped to memory.

    Another possibility, suitable for a machine with many CPU registers (x64, RISC V, ARM), might be to do register allocation of local variables both in and across functions and their nested functions.  Thus, just as local variables may live in registers, non-local variables might also.  This would also preclude recursion, but would make a performance difference.

    Lastly, let's mention inlining as that accomplishes a lot of the efficiency we expect from machines with large numbers of CPU registers, by avoiding use of memory for local variables — once two functions are merged by inlining, they share the same local variable storage space; inlining can even remove the pair of address taken by caller and then dereference within callee, allowing to map such a local variable to a register, instead of memory as would be needed if the address actually had to be materialized.  As inlining is done by analysis of the functions by the compiler, it is flexible in allow one function to call another, including recursion — though certain constructs may disable some optimization.