A while ago I posted this question regarding strange behavior I was experiencing in trying to step through a MASM program.
Essentially, given the following code:
; Tell MASM to use the Intel 80386 instruction set.
.386
; Flat memory model, and Win 32 calling convention
.MODEL FLAT, STDCALL
; Treat labels as case-sensitive (required for windows.inc)
OPTION CaseMap:None
include windows.inc
include masm32.inc
include user32.inc
include kernel32.inc
include macros.asm
includelib masm32.lib
includelib user32.lib
includelib kernel32.lib
.DATA
BadText db "Error...", 0
GoodText db "Excellent!", 0
.CODE
main PROC
int 3
mov eax, 6
xor eax, eax
_label: add eax, ecx
dec ecx
jnz _label
cmp eax, 21
jz _good
_bad: invoke StdOut, addr BadText
jmp _quit
_good: invoke StdOut, addr GoodText
_quit: invoke ExitProcess, 0
main ENDP
END main
I could not get the int 3
instruction to trigger. It was clear why it didn't, examining the disassembly:
00400FFD add byte ptr [eax],al
00400FFF add ah,cl
--- [User path]\main.asm
mov eax, 6
00401001 mov eax,6
xor eax, eax
00401006 xor eax,eax
_label: add eax, ecx
The int 3
instruction had been replaced with add al,cl
, but I had no idea why. I managed to track the problem to whether or not Incremental Linking was enabled. The above disassembly was generated with Incremental Linking disabled (/INCREMENTAL:NO option on the command line). Re-enabling it would result in something like the following:
.CODE
main PROC
int 3
00401010 int 3
mov eax, 6
00401011 mov eax,6
xor eax, eax
00401016 xor eax,eax
I should note that the interleaving lines are references back to the original code (I guess a feature of Visual Studio's disassembly window). With Incremental Linking enabled, the disassembly corresponds exactly to what I had written in the program, which is how I expected it to behave all along.
So, why would disabling Incremental Linking cause the disassembly of my program to be altered? What could be happening behind the scenes that would actually alter how the program executes?
The "add" instruction is a two byte instruction, the second of which is the 1-byte opcode of your int3. The first byte of the two byte add instruction is probably some garbage just before the entrypoint. The address of the add instruction is probably 1 byte before where the int3 instruction would be.
I quickly assembled and then disassembled those two instructions with GNU as en objdump, and the result is:
8: 00 cc add %cl,%ah
a: cc int3
Here you can clearly see that the the add instruction contains the second byte 0xcc, while int3 is 0xcc
IOW make sure that you start disassembling on the entry point to avoid this problem.