Why is the address where the stack begins predictable enough to allow repeatable exploitation but appears to be unpredictable?

Question

I've been teaching myself to write exploits and shellcode recently and many of the guides/books make the claim that we can guess where the stack begins and thereby with a little work guess what address on the stack our shellcode starts at.

• Why is the address of the start of the stack predictable? Shouldn't the stack start at some random address in memory?
• If the address start of the stack is just some fixed offset in virtual address space then why does it differ from machine to machine?
• What is the margin of error between different platforms and the address of the start of the stack?

Related: "the stack begins with the same address for every program", Address woes from Hacking: The Art of Exploitation, Why does the stack address grow towards decreasing memory addresses?

EDIT For my tests Address Space Layout Randomization (ASLR) is turned off.

Answer 1

In my mind, determining the start address of the stack being predictable isn't as important as being able to modify the top of the stack. When a method is called, various registers are stored on the stack, including the program counter which points to the location where the code will resume executing when the method returns. In addition, stack space is reserved to store the method's local variables. (After all, they need to be stored somewhere!) These variables can be used in some cases to overwrite the top of the stack to change the location where code resumes execution (for example). Some disassembling and examining memory with a debugger is generally sufficient to determine where the variables end up on the stack in relation to the program counter. Then, it's a "simple" matter of overwriting the return location in the stack to get it to return somewhere else when the method returns.

This doesn't answer your second two points, and I think only sort-of answers your first, but it's a start.