How to simplify Assembly Translation Shift Right by 32 Xor Absolute Number And Value

Question

I don't know the original code but I don't believe it's this complicated with right shift's and abs.

Here is how the decompiled IDA PRO code renamed looks like

char Ship; //Could be 0-7 (8 is reversed for special purpose)
char NewShip = 1; //Could be 0-7  (8 is reversed for special purpose)
short Frequency = 0; //This could be from 0 to 9999
bool NumberToFrequency = true;

Frequency = GetNextFrequencyToJoin(player->MyArena);
if ( NumberToFrequency )
{ //TODO: maybe the below is just Frequency % 7; ?
  NewShip = (((unsigned long)Frequency >> 32) ^ abs(Frequency) & 7) - ((unsigned long)Frequency >> 32);
  Ship = NewShip;
} else {
  Ship = NewShip;
}

Here is a IDEOne test http://ideone.com/Q2bEjU

Seems NewShip = abs(frequency) & 7); is all I really need it seems I tested all the possibilities by looping it never screws up.

Another decompiler gave me this result

 asm("cdq ");
 NewShip = ((Var1 ^ Var2) - Var2 & 7 ^ Var2) - Var2;

Which has no right shifts or anything still looks alien to me, probably shows how absolute number works and still no clue where right shift 32's came from.

Whats the NumberToFrequency suppose to do is make the frequency the same as the ship but of course the frequency goes past 7 so the remaining values should still translate to Ship values so I assume it's just a modulus % of 7.

But why such a complicated code maybe it means something completely different? I'm just asking what the code means. I'll add the assembly code below. I can't even find the Shift right 32 in the assembly below I'm pretty sure it's in same place.

.text:0040DD3A                 mov     ecx, [ebp+1Ch]  ; arena
.text:0040DD3D                 call    GetNextFrequencyToJoin
.text:0040DD42                 mov     ecx, [ebp+1Ch]
.text:0040DD45                 mov     si, ax
.text:0040DD48                 mov     [esp+220h+var_20C], si
.text:0040DD4D                 cmp     [ecx+1ACCEh], ebx
.text:0040DD53                 jz      short loc_40DD98
.text:0040DD55                 movsx   eax, si
.text:0040DD58                 cdq
.text:0040DD59                 xor     eax, edx
.text:0040DD5B                 sub     eax, edx
.text:0040DD5D                 and     eax, 7
.text:0040DD60                 xor     eax, edx
.text:0040DD62                 sub     eax, edx
.text:0040DD64                 mov     [esp+220h+var_20F], al

EDIT: I found the answer on my own seems those shift 32 >> 32 are useless garbage added for some old C compiling support with types to match 32 bit DWORD's or some crap like that.

Answer 1

The shifts are not useless. It is a form of banchless logic that Hexray did not manage to reproduce in its c disassembly.

.text:0040DD55                 movsx   eax, si
.text:0040DD58                 cdq
.text:0040DD59                 xor     eax, edx
.text:0040DD5B                 sub     eax, edx
.text:0040DD5D                 and     eax, 7
.text:0040DD60                 xor     eax, edx
.text:0040DD62                 sub     eax, edx

Is the significant code. EDX:EAX is the sign-extended version of SI, so EDX is either 0 or -1. The xor either leaves eax untouched or inverts it, the sub leaves it untouched or adds one and so on in total:

if (si < 0) {
    eax = ~si;
    eax += 1;
    eax &= 0x7;
    eax = ~eax;
    eax += 1;
} else {
    eax = si & 0x7;
}

The first branch can still be simplified, but I leave that to you...

---

update

That the branches only differ for si<0 already gives a hint at what is happening. The sequence eax = ~si; eax += 1; can be understood as the two's-complement, so inserting our knowledge of this complement we get

if (si < 0) {
    eax = -1 * si;
    eax &= 0x7;
    eax *= -1;
} else {
    eax = si & 0x7;
}

or in short

eax = (abs(si) & 0x7) * sign(si);

Or with the signed modulus operator

al = si % 8;