calloc 2 gb of memory failed for type of 'char' but succeed for 'short'

Question

I tried to calloc a table 2GByte large. The following code failed

#define MAX_FILEID      131072                  // 17 bits 131K file IDs
#define MAX_OFFSET      8192                    // offset loctions (refLoc - Loc)
#define MATCH_TAB_SIZE  (MAX_OFFSET*MAX_FILEID)

matchTab = (unsigned char*) calloc(MATCH_TAB_SIZE*2, sizeof(unsigned char));
if ( matchTab==NULL)
{
    fprintf(stderr, "calloc %f Mbyte failed for matchTab \n", MATCH_TAB_SIZE*sizeof(unsigned char)/(1024*1024.0) );
    return;
}
else
{
    fprintf(stdout, "assigned %f Mbyte \n", MATCH_TAB_SIZE*sizeof(unsigned char)/(1024*1024.0) );
}

However if I replace the calloc line to

matchTab = (unsigned short*) calloc(MATCH_TAB_SIZE, sizeof(unsigned short));

It succeeded.

I am wondering why. My machine is a 64bit linux with at least 80 gb ram.

Answer 1

8192*131072*2 = 2³¹

2³¹ > INT_MAX in your implementation, so the above expression overflows. The result of it is undefined.

Use literals of a type that's large enough, such as unsigned int:

#define MAX_FILEID      131072U                  // 17 bits 131K file IDs
#define MAX_OFFSET      8192U                    // offset loctions (refLoc - Loc)
#define MATCH_TAB_SIZE  (MAX_OFFSET*MAX_FILEID)

matchTab = calloc(MATCH_TAB_SIZE*2U, sizeof(unsigned char));

Alternatively, cast the numbers to size_t while they are small:

matchTab = calloc(((size_t)MATCH_TAB_SIZE)*2, sizeof(unsigned char));

Note, don't cast the result of calloc and friends. sizeof(unsigned char) is always 1 so you can use 1 instead if you want.