Two years later and I'm back at it. Trying to tackle the shellcoders handbook again, but I continue to find inconsistencies. The book provides the following function:
int triangle (int width, in height){
int array[5] = {0,1,2,3,4};
int area;
area = width * height/2;
return (area);
}
and the following disassembly for the function:
0x8048430 <triangle>: push %ebp
0x8048431 <triangle+1>: mov %esp, %ebp
0x8048433 <triangle+3>: push %edi
0x8048434 <triangle+4>: push %esi
0x8048435 <triangle+5>: sub $0x30,%esp
0x8048438 <triangle+8>: lea 0xffffffd8(%ebp), %edi
0x804843b <triangle+11>: mov $0x8049508,%esi
0x8048440 <triangle+16>: cld
0x8048441 <triangle+17>: mov $0x30,%esp
0x8048446 <triangle+22>: repz movsl %ds:( %esi), %es:( %edi)
0x8048448 <triangle+24>: mov 0x8(%ebp),%eax
0x804844b <triangle+27>: mov %eax,%edx
0x804844d <triangle+29>: imul 0xc(%ebp),%edx
0x8048451 <triangle+33>: mov %edx,%eax
0x8048453 <triangle+35>: sar $0x1f,%eax
0x8048456 <triangle+38>: shr $0x1f,%eax
0x8048459 <triangle+41>: lea (%eax, %edx, 1), %eax
0x804845c <triangle+44>: sar %eax
0x804845e <triangle+46>: mov %eax,0xffffffd4(%ebp)
0x8048461 <triangle+49>: mov 0xffffffd4(%ebp),%eax
0x8048464 <triangle+52>: mov %eax,%eax
0x8048466 <triangle+54>: add $0x30,%esp
0x8048469 <triangle+57>: pop %esi
0x804846a <triangle+58>: pop %edi
0x804846b <triangle+59> pop %ebp
0x804846c <triangle+60>: ret
For academic reasons I'm trying to break down and explain each line of the assembly. But several things just feel wrong, for example: lea 0xffffffd8(%ebp), %edi, it is my understanding that the first part means multiply the base pointer by 0xffffffd8, that seems incorrect. Another example is mov $0x30, $esp, why would you move a literal value into the stack pointer register. I could understand if it was mov $0x30, (%ebp), but that does not seem to be the case. Am I wrong, or does this all seem just wrong?
But several things just feel wrong
Yes, they are. It isn't unusual for a book to have typographical errors. Usually, you should look for a published list of errata when you see something that makes you scratch your head. The publisher's website is a good place to look, as is the author's. I don't know what the book's exact title is, so I can't search for it myself, but you should be able to find it easily.
Of course, it may not be that simple. Books by less-reputable publishers often won't make an errata list available, and less-popular books often don't have enough readers to have caught the errors. You can do your part by locating the author's email address and notifying them of the errors you've found. Or, if you aren't sure whether they're errors, asking the author to clarify. (You don't want to expect the author to provide a personal tutorial for you, but specific questions about things published in their books are always fair game.)
lea 0xffffffd8(%ebp), %edi, it is my understanding that the first part means multiply the base pointer by0xffffffd8, that seems incorrect
In this case, it is your understanding of what the code does that's incorrect. I blame this demented AT&T syntax. The translation to Intel syntax is:
lea edi, DWORD [ebp + ffffffd8h]
which is equivalent to:
lea edi, DWORD [ebp - 28h]
So this is actually equivalent to:
mov edi, ebp
sub edi, 28h
Now, you're right that the LEA instruction can do multiplication. Well, kind of. It can scale by certain constants, like 2, 4, and 8, which has the same effect as a multiplication. But this form isn't encoding a multiplication (or, again, more accurately, it is a scale by 1).
mov $0x30, $esp, why would you move a literal value into the stack pointer register. I could understand if it wasmov $0x30, (%ebp), but that does not seem to be the case.
Yeah, moving a literal into the stack pointer is a very strange thing to do. Never say never, but this should scream "bug" (or "typo").
But look at the next instruction:
repz movsl %ds:(%esi), %es:(%edi)
The repeat string operation prefixes cause a string instruction (in this case, MOVSL) to be repeated the number of times specified in the ECX register, so that previous instruction was probably supposed to initialize ECX. It would make sense to initialize ECX to 30h, since that's the amount of space that was previously allocated on the stack (subl $0x30, %esp).
But there's another error here: the REPZ (or, equivalent REPE) prefix doesn't make sense with the MOVS instruction! The [N]Z/[N]E usually means that the zero flag is used as a secondary termination condition, but a move doesn't set any flags, so it doesn't make sense to write REPZ MOVS! That should just be REP MOVS.
Frankly, the entire disassembly is suspect, as far as I'm concerned. I begin to wonder if the book is even worth the paper it's written on. Why would you be showing unoptimized assembly code? If you're trying to learn assembly language, you don't want to learn how to write sub-optimal code. If you're trying to learn reverse-engineering, it doesn't make sense to study unoptimized code because that's not what a compiler will generate. Same thing for exploits. I can't think of a good reason why you would ever want to waste your time looking at unoptimized code. There's just a bunch of distracting noise that isn't teaching you anything useful.
For example, you see the telltale sign of unoptimized code at the very beginning: not eliding the initialization of the base pointer (EBP).
The purpose of the REPZ MOVS instruction (and associated required instructions) is also a complete mystery to me. I can't even see a reason why a compiler would generate these with optimizations disabled.
I guess the author had to turn off optimization, though, because otherwise that entire "array" allocation/initialization would have been elided. Not the best of examples.
This sequence also has to be a mistake:
sar $0x1f, %eax shr $0x1f, %eax
An unsigned right-shift by 31 makes sense (isolating the sign bit as part of an optimized signed division by 2), but doing it right after a signed right-shift doesn't. (The expected sar %eax that is part of this optimized division comes later, in the typical GAS format that omits the $1 immediate.)
If all (or even most) of the code is like this, my recommendation is either to give up on that book and find a different one, or compile and disassemble the C functions yourself.
A non-broken C compiler would generate the following code for that C function:
; Load second parameter from stack into EAX.
movl 8(%esp), %eax
; Multiply that second parameter by the first parameter.
; (Could just as well have used a second movl, and then done a reg-reg imull.)
imull 4(%esp), %eax
; Make a copy of that result in EDX.
movl %eax, %edx
; Optimized signed divide-by-2:
shrl $31, %eax
addl %edx, %eax
sarl $1, %eax ; GAS encodes this as 'sarl %eax', making the $1 implicit
ret
Or, if optimizations were disabled (this is a bit more variable among different compilers, another reason why looking at unoptimized code is silly, but you can get the basic idea):
; Set up a stack frame
pushl %ebp
movl %esp, %ebp
; Allocate space on the stack for the pointless "array" array,
; and store the values in that space.
; (Why 32 bytes instead of only 30? To keep the stack pointer aligned.)
subl $32, %esp
movl $0, -24(%ebp)
movl $1, -20(%ebp)
movl $2, -16(%ebp)
movl $3, -12(%ebp)
movl $4, -8(%ebp)
; Get first parameter from the stack.
movl 8(%ebp), %eax
; Multiply it by the second parameter.
imull 12(%ebp), %eax
; Make a copy of the result.
movl %eax, %edx
; Optimized signed divide-by-2 (some compilers will always apply this
; strength-reduction, even when optimizations are disabled; others won't
; and will go ahead and emit the IDIV instruction you might expect):
shrl $31, %edx
addl %edx, %eax
sarl $1, %eax
; Store the result in a temporary location in memory.
movl %eax, -4(%ebp)
; Then load it back into EAX so it can be returned.
movl -4(%ebp), %eax
; Tear down the stack frame and deallocate stack space.
leave
ret