This question has been edited to clarify it.
I have the following matrix, defined in page 1 of [reteam.org/papers/e59.pdf], written in R notation:
m1 = matrix(c(207560540,956631177,1,956631177,2037688522,1,2037688522,1509348670,1),ncol=3,byrow=T)
The determinant of m1 should be a integer multiple of 2^31 -1.
As indicated in accepted answer, det(m1) should be -1564448852668574749.
However, in R, I got
> det(m1)
[1] -1564448852668573184
and, using a simple equation by hand:
> m1[1,1]*(m1[2,2]-m1[3,2]) - m1[2,1]*(m1[1,2] - m1[3,2]) + m1[3,1]*(m1[1,2]- m1[2,2])
[1] -1564448852668574720
As indicated in accepted answer, the correct determinant is obtained and checked by:
#include <inttypes.h>
#include <stdio.h>
int main() {
int64_t m1[3][3] = {{INT64_C(207560540) , INT64_C(956631177) , INT64_C(1)},{ INT64_C(956631177), INT64_C(2037688522), INT64_C(1)},{INT64_C(2037688522), INT64_C(1509348670) , INT64_C(1)}};
int64_t dm1 = m1[0][0]*(m1[1][1]-m1[2][1]) - m1[1][0]*(m1[0][1] - m1[2][1]) + m1[2][0]*(m1[0][1]- m1[1][1]);
int64_t divisor = (INT64_C(1)<<31) -1;
int64_t tmp = dm1/divisor;
int64_t check = tmp * divisor;
printf("dm1 == %" PRIu64"\n",dm1);
printf("(dm1/(2^31 -1))* %" PRIu64 " == %" PRIu64 "\n", divisor, check);
}
The following text is the old question. The main error was using a unsigned type.
My old minimum non-working code example was:
#include <inttypes.h>
#include <stdio.h>
int main() {
uint64_t dm1 = 1564448852668573184;
uint64_t divisor = (UINT64_C(1)<<31) -1; //powl(2,31)-1;
uint64_t tmp = dm1/divisor;
uint64_t check = tmp*divisor;
printf("dm1 == %" PRIu64"\n",dm1);
printf("(dm1/(2^31 -1))* %" PRIu64 " == %" PRIu64 "\n", divisor, check);
}
Its output is
dm1 == 1564448852668573184
(dm1/(2^31 -1))* 2147483647 == 1564448850521091102
The problem is that the value of the second line should be equal to the one in the first line.
What is my mistake? How can I make this work?
The numerator is not an exact multiple of the divisor, so there is a remainder, and the quotient is truncated.
1564448852668573184 / 2147483647 = 728503266 remainder 2147482082
Multiplying back,
2147483647 * 728503266 + 2147482082 = 1564448852668573184
EDIT:
The determinant of the 3x3 matrix shown in your linked reference is -1564448852668574749. This is exactly divisible by 2147483647 to give -728503267.
So you have an arithmetic overflow somewhere.
ANSWER:
The value of the matrix determinant in your linked example is negative. Please use int64_t instead of uint64_t.