BSD memcmp(3) difference between manual and implementation

Question

According to man memcmp on OSX Darwin:

The memcmp() function returns zero if the two strings are identical, otherwise returns the difference between the first two differing bytes (treated as unsigned char values, so that \200 is greater than \0, for example). Zero-length strings are always identical. This behavior is not required by C and portable code should only depend on the sign of the returned value.

However, when I test this:

#include <stdio.h>
#include <string.h>

int main()
{
    printf("%i\n", memcmp("\200", "\0", 1));
    return (0);
}

It displays -1, which would indicate that \200 is less than \0.

Is there any explanation for this?

The compiler version according to gcc --version is "Apple LLVM version 9.0.0 (clang-900.0.39.2)" and the system is running High Sierra 10.13.4

Answer 1

This is a compiler bug. The compiler incorrectly evaluates calls to memcmp when both arguments are literals. When memcmp is actually called, it returns the expected result.

The following was tested with Apple LLVM version 9.1.0 (clang-902.0.39.1) on macOS 10.13.4 (17E199). I compiled with “clang -std=c11”, with either “-O0” or “-O3” to select the optimization level, and with “-S” to generate assembly.

Consider four alternative calls to memcmp:

    printf("%i\n", memcmp("\200", "\0", 1));

    printf("%i\n", memcmp((char[] ) { '\200' }, "\0", 1));

    printf("%i\n", memcmp((unsigned char[] ) { '\200' }, "\0", 1));

    char a[1] = { 128 };
    char b[1] = { 0 };
    printf("%i\n", memcmp(a, b, 1));

For the first two calls, the compiler generates incorrect assembly that passes a hardcoded value of −1 to printf. There is no call to memcmp; it has been optimized away, even in the “-O0” version. (In the “-O0” versions, the −1 is encoded as 4294967295, which is equivalent in its context.) When memcmp is called with string literals or compound literals, its return value is known at compile time, so the compiler has evaluated it. However, it has done so incorrectly.

For the third call, the compiler generates incorrect assembly that passes a hardcoded value of 1. This suggests the compiler is (incorrectly) using the type of the literal in its evaluation.

For the fourth call, in which we use defined objects that are not literals, the “-O0” version calls memcmp. When run, the program prints the correct result, 128. For the “-O3” version, the compiler generates correct assembly with a hardcoded value of 128. So the compiler does have an algorithm that correctly evaluates a memcmp at compile time, but it used a different, faulty algorithm for the cases with the literals.

When one literal and one non-literal are used, the compiler generates correct code. This explains why this bug has not been seen and fixed previously: Calls to memcmp with two literals are rare, and code that both does that and depends on the magnitude of the result or uses characters having high bits set is rarer.

(I reported the bug to Apple.)