so Im doing a variation of the binary bomb. Heres what the phase is
0x0000000000401205 <+0>: sub $0x8,%rsp
0x0000000000401209 <+4>: cmp $0x3,%rdi #rdi = 3
0x000000000040120d <+8>: je 0x40121d <phase_2+24> #if(rdi == 3) skip explosion
0x000000000040120f <+10>: callq 0x401c01 <bomb_ignition>
0x0000000000401214 <+15>: mov $0xffffffffffffffff,%rax
0x000000000040121b <+22>: jmp 0x40124c <phase_2+71>
0x000000000040121d <+24>: xor $0xffffffffffffff1c,%rsi #rsi ^= 0xffffffffffffff1c
0x0000000000401224 <+31>: and %rsi,%rdx #rdx &= rsi
0x0000000000401227 <+34>: add $0x1c8,%rdx #rdx += 456
0x000000000040122e <+41>: cmp $0x2c5,%rdx #rdx == 709
0x0000000000401235 <+48>: sete %al
0x0000000000401238 <+51>: movzbl %al,%eax
0x000000000040123b <+54>: cmp %rcx,%rdx #rdx == rcx
0x000000000040123e <+57>: je 0x40124c <phase_2+71>
0x0000000000401240 <+59>: callq 0x401c01 <bomb_ignition>
0x0000000000401245 <+64>: mov $0xffffffffffffffff,%rax
0x000000000040124c <+71>: add $0x8,%rsp
0x0000000000401250 <+75>: retq
Ive annotated what i think it means next to the asm. What Im stuck on is <+24> where it xors $0xffffffffffffff1c and %rsi. %rsi is the second parameter passed correct? So how do I figure out what it is so i can xor it? Because it then ands it to %rdx which is the third parameter.
So what I think it does it take 4 numbers. 1st is '3' (right?). Then xor the second with 0xffffffffffffff1c. Ands this to the third number. Then third number adds 456, and should be 709. Which is what the 4th number should be. Also Im still a bit confused on sete and movzbl. But I cant figure out what the second and third number should be. Am I correct in my prediction? ANY HELP APPRECIATED. Thanks
Does the caller check the return value? I'm assuming so, otherwise you could choose whatever you want for rsi and rdx, and then calculate what rcx will match. It sets rax = -1 (bomb explosion), or 0 or 1 (sete/movzx) based on the cmp $0x2c5,%rdx result.
Lets name our function inputs so we can talk about the function args separately from the current value in a register: x:rsi, y:rdx, and z:rcx. (Ignoring the first arg in rdi, which is checked separately).
z only interacts with the others once, so it can be chosen freely.
rsi ^= -228; // 0xffffffffffffff1c
rdx &= rsi; // rdx &= (x ^ -228)
rdx += 456
if(rdx != rcx) explode();
We're assuming we also need rdx == 709 = 0x2c5 (and thus z = 709, too). Let's work backwards from rdx == 709:
709 - 456 = 253 // rdx before add $0x1c8,%rdx
Then we need x and y such that y & (x^-228) == 253. This is one equation in 2 variables, so there should be multiple solutions. The "simplest" would be y = 253 (0xfd), and then choose an x that will leave all of those bits set.
For example, choose x so that x ^ 0x1c == 0xfd as well. (We can ignore the high bits, because AND with 0xfd clears them). You can solve this equation like you would for addition. xor is its own inverse (and is commutative/associative), so we can xor both sides by 0x1c to solve for x = 0xfd ^ 0x1c = 0xe1 = 225.
@Prl's solution uses x = 0xe1 but y=255. This is fine; extra set bits are wiped away by the AND.