I have been doing some research into inline assembly in C and how the call stack works but I have been unable to figure out if it's at all possible to retrieve the address of a variable that is requesting the return value of a function, from WITHIN the function.
int hypothetical_func(){
/*...
.. some assembly to get the address of 'int a' from the call stack?
...*/
return 5;
}
int main(){
int a = hypothetical_func();
}
Is this at all possible?
NO. int is returned in a register, and the callee has no involvement in what the caller does with that register after it returns. It might never be stored in memory.
If the return-type wasn't int, but instead something large enough that the calling convention returned it by value, then hypothetical_func would have an address for an output. (Or a hypothetical (terrible) calling convention might return even int via hidden pointer instead of a register. Assuming the machine is a register machine like all real CPUs.)
But that might just be a return-value temporary, not the actual LHS of an assignment. (Or initialization, which is close enough to the same thing in C, if not C++). Especially if the assignment is to a global or something. See What prevents the usage of a function argument as hidden pointer? for the case of *out = foo(); where T *out is a function arg. It's highly non-trivial to prove if/when it's safe to pass that function arg along as the return-value object for foo().
And some compilers don't even try to optimize, and just make space on the stack for the return-value temporary and copy from there into the final object.
And as @prl points out, the return-value might not even be the initializer for a variable. e.g. printf("%d\n", foo()); just passes on the return value to a function arg. Or foo(); discards the return value, not assigning it anywhere. (But if the calling convention specifies that the function returns by hidden pointer, the caller must pass a pointer to enough scratch space. The callee is still going to write its return value and needs to not segfault from a bad pointer or overwrite something else. That's an asm / calling-convention detail separate from the operation of the C abstract machine. Or I guess you could say the return-value object still exists, it's just not assigned anywhere.)
Plus with inline assembly, you don't even have access to that. Unless you count writing a __attribute__((naked)) function where you still write the whole function inside an asm statement, and the compiler doesn't handle anything except the name-mangling of the function name. No prologue or epilogue, or abstracting away the calling convention with C variables for args and one that you return. (/grumble that C compilers can't create functions that return multiple separate values in multiple registers like you can in hand-written asm.)
But even with hand-written asm, there's no way to do this for normal calling conventions on normal ISAs like x86 and ARM. The return-value object for int is just a register.