I am having some trouble understanding how the esp and ebp registers are used.
Why do we do:
pushl %ebp
movl %esp, %ebp
at the start of every function? What is ebp holding when it is pushed for the first time?
At the start of every function ebp is pointing wherever the calling function wanted it, it's not relevant to the current function until the code for the current function chooses to use it. ebp is just a stack frame pointer in case you choose to have a stack frame. The notion is that YOU CAN use ebp to have a non-moving reference to the stack for your function while you are free to continue to add or remove items on the stack using esp. If you were to not use a stack pointer and were to continue to use esp as the reference to the stack then where a particular item on the stack is over the course of your function varies relative to esp. If you set ebp before you start using the stack (other than to save ebp) then you have a fixed relative address to the parameters on the stack that your function cares about, like passed parameters, local variables, etc.
You are perfectly free to use eax or edx or any other register as a stack frame pointer within your function, ebp is there as a general purpose register for you to use for stack frames since x86 has historically had a stack dependency (return addresses, and old calling conventions were stack based). Other instruction sets with more registers might simply choose a register for the compiler implementation as the function pointer/stack frame pointer. If you have the option and choose to use a stack frame. It burns a register you could be using for other things, burns more code and execution time. Like using other general purpose registers, ebp is non volatile per the calling conventions being used today, you need to preserve it and return it the way you found it. So what it points to is specific to the function. What it pointed to when your function was entered was specific to the calling function.
A particular compiler implementation may choose to have stack frames and may choose how it uses ebp. And if it is always used the same way when enabled then with that toolchain you might have a debugger or other tool that can take advantage of that. For example if the first thing in the function is to push ebp on the stack then the return address to the calling function within any function relative to ebp is fixed (well unless there was some tail optimization then maybe its the caller of the caller (of the caller (of the caller))). You are burning a register and stack space and code space for this feature but, like compiling for debugging you can compile with a stack frame during development to use these features.
The reason why you start with a push is that is a good way to use the frame pointer and define a consistent location. Pushing it on the stack as the first thing you do 1) preserves ebp so you don't crash the calling function(s) 2) defines a consistent reference point addresses below ebp are the return address and calling parameters at a fixed offset for the duration of the function. Local variables are at fixed addresses above ebp for a scheme like this. Compilers, as well as humans, are more than capable of not needing to do this, my first parameter might be at esp-20 at one point in the code and I may then go push 8 bytes more on the stack now that same parameter is at esp-28, just code it as such.
But for debugging purposes debugging the code produced and at times for example finding the return address at a fixed offset. Burning another register, is IMO lazy but, can definitely help debug and increase the quality of the compiler output. Find bugs in the output of the compiler faster, and help folks trying to read the code understand it faster with less effort. With a stack frame pointer used properly all parameters and local variables are at a fixed offset to the stack frame pointer through the duration of the function between the points where the stack frame pointer is setup and cleaned up. Push pointer to save it set frame pointer to stack pointer with or without an offset. To the pop of the frame pointer before a return.