AVR C hex and dec multiplication not as expected

Question

I just discovered a bug in my code when multiplying hex integers (ex. 0xFFFF) with dec integers (ex. 2). This is the code where the problem occurs:

print_int_new_line(0xFFFF*2);
print_int_new_line(65535*2);

Executing this code gives me the following result:

65534
131070

This is relevant UART code:

void print_int_new_line(uint64_t data) {
    print_int(data);

    print_new_line();
}

void print_int(uint64_t data) {
    char data_buffer[(int)(log(data)+2)]; // size of the number (log(number)+1) + 1 for the /0

    ultoa(data, data_buffer, 10); // convert the int to string, base 10

    // print the newly created string
    print_string(data_buffer);
}

void print_string(char * data) {

    // transmit the data char by char
    for(; *data != '\0'; data++){
        USART_transmit(data);
    }
}

void USART_transmit(const char * data){
    /* Wait for empty transmit buffer */
    while ( !( UCSR0A & (1<<UDRE0)) )
    ;

    /* Put data into buffer, sends the data */
    UDR0 = *data;
}

Some info about my setup:

MCU: ATmega2560 Board: Arduino Mega2560 UART baudrate: 38400 IDE: Atmel Studio 7.0.4.1417

Using the AVR toolchain.

I read on this stackoverflow page that multiplication is possible between hex and dec ints. Also, testing this in an online c compiler gives the correct output.

Can anyone give me an explanation?

Answer 1

This behavior is due to differences in handling decimal and hexadecimal integer constants.

For both 0xFFFF and 65535, the compiler will first try to convert the value to an int. But since the platform has a 16-bit int type where INT_MAX is 32767, that conversion cannot be performed.

The key difference is the next step. For the hexadecimal constant 0xFFFF, the compiler will try to convert it to an unsigned int, which it does to the equivalent of (unsigned int)65535. But, for the decimal constants, conversions to unsigned types are not attempted. The next conversion attempt is to long int. This succeeds and is equivalent to (long int)65535.

So the calls to print_int_new_line are equivalent to:

print_int_new_line((unsigned int)65535*2);
print_int_new_line((long int)65535*2);

And when 2 is promoted to do the multiplication:

print_int_new_line((unsigned int)65535*(unsigned int)2);
print_int_new_line((long int)65535*(long int)2);

The unsigned int result of the first multiplication is too small to hold the full result, so it is truncated to 65534. The long int can hold the result, so it produces the correct answer of 131070.

You can force the hexadecimal constant to use a long int by appending an L (i.e. 0xFFFFL).