I want to write a x86 masm program that calculates a big factorials (results up to 128 bits), but I am not sure how to do that. I've written a version that calculates up to 32bits, how could I change it up so that it works for larger numbers?
128 bits is too big for mul instruction, which saves only 64-bits result (EDX (32bits) + EAX (32bits)).
So first of all you need 4 DWORDS of memory to store 128bits result.
Then you should replace your mul ebx;
on some procedure call, that can handle 128bits multiplication. The code can be something like this:
.data
;...
bignum dword 1, 0, 0, 0; initial answer = 0000 0000 0000 0001h
....
call largemul; replace for mul ebx
....
largemul proc;
mov ebx, 10 ;EBX = multiplier
mov eax, [bignum]
mul ebx ;EDX:EAX = EAX*EBX
mov [bignum], eax ;save result
mov ecx, edx ;save carried part in ECX
mov eax, [bignum+4]
mul ebx
add eax, ecx ;add carried part from previous multiplication
mov [bignum+4], eax
mov ecx, edx
mov eax, [bignum+8]
mul ebx
add eax, ecx
mov [bignum+8], eax
mov eax, [bignum+8]
mul ebx
add eax, ecx
mov [bignum+12], eax
largemul endp;