We had a crash today caused by an arithmetic exception in the idiv instruction in the following x86 assembly.
mov r10d,DWORD PTR [rbp+0x70]
xor ecx,ecx
mov eax,r15d
test r10d,r10d
setle cl
cdq
idiv ecx
Here are the values of all the registers in gdb:
rax 0x64 100
rbx 0xaebc30 11451440
rcx 0x0 0
rdx 0x0 0
rsi 0x1 1
rdi 0xaebc88 11451528
rbp 0x7fa56809b840 0x7fa56809b840
rsp 0x7fa56effc080 0x7fa56effc080
r8 0x12 18
r9 0x100016a0000a72e 72059148816131886
r10 0x1 1
r11 0x0 0
r12 0x9e2bb8 10365880
r13 0x0 0
r14 0xaebc80 11451520
r15 0x64 100
rip 0x495ebb 0x495ebb
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
Since the zero flag is set, setle cl writes 0 into ecx, which causes a divide by zero. The thing I don't understand is how the zero flag is set in the first place. The value of r10 is 1, so to my knowledge test r10d,r10d should unset it. cdq doesn't appear to modify it so I'm not sure what's happened here.
Can anyone with more x86 experience understand what the problem is? Is more information needed?
And this register dump is from right before idiv (or after GDB catches the SIGFPE which should be the same thing)?
You have it backwards: ZF=1 means the "equals" condition is true. (So less-or-equal setle is also true). So setle cl should give you a 1.
With R10 = 1, R10D = 1 also. R10D & R10D != 0 so ZF should be cleared. Thus r10d is NOT <= 0, so setle puts a 0 in CL and you do get a #DE divide exception.
With this NASM source code:
mov r10d, 1
xor ecx,ecx
mov eax, 0x64
test r10d,r10d
setle cl
cdq
idiv ecx
Built into a Linux static executable with nasm -felf64 foo.asm && ld foo.o -o foo (the entry point defaults to the top of the .text section), I ran it under GDB (because that was quicker than thinking through all the details, and I wanted to see if EFLAGS were changed by the faulting idiv).
Use GDB's starti command to start at the top, and single-step. (With layout reg).
Right before idiv, I get EFLAGS = [ IF ], ecx = 0. (With r10d = 1).
As expected, an R10D > 0 (not <= 0) leads to ECX = 0.
After trying to step past the idiv, or using continue, EFLAGS = [ RF IF ]. So your GDB register dump doesn't make sense, unless a different CPU or OS can do something different with FLAGS on a fault.