I have this simple C program:
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
int main (int argc, char **argv) {
int i = 0;
int j = 0;
size_t size = 4194304; /* 4 MiB */
char *buffer = malloc(size);
char *buffers[10] = {NULL};
void *tmp_pointer = NULL;
fprintf(stderr, "initial size == %zu\n", size);
fprintf(stderr, "initial buffer == %p\n\n", buffer);
srand(time(NULL));
/* let's see when it fails ... */
for (; i < 10; ++i) {
/* some random writes */
for (j = 0; j < 1000; ++j) {
buffer[rand() % size] = (char)(rand());
}
/* some interleaving memory allocations */
buffers[i] = malloc(1048576); /* 1 MiB */
size *= 2;
fprintf(stderr, "new size == %zu\n", size);
tmp_pointer = realloc(buffer, size);
if ((tmp_pointer == NULL) || (errno != 0)) {
fprintf(stderr, "tmp_pointer == %p\n", tmp_pointer);
fprintf(stderr, "errno == %d\n", errno);
perror("realloc");
return (1);
} else {
buffer = tmp_pointer;
}
fprintf(stderr, "new buffer == %p\n\n", buffer);
}
fprintf(stderr, "Trying to free the buffers.\n");
free(buffer);
if (errno != 0) {
fprintf(stderr, "errno == %d\n", errno);
perror("free(buffer)");
return (2);
}
for (i = 0; i < 10; ++i) {
free(buffers[i]);
if (errno != 0) {
fprintf(stderr, "i == %d\n", i);
fprintf(stderr, "errno == %d\n", errno);
perror("free(buffers)");
return (3);
}
}
fprintf(stderr, "Successfully freed.\n");
return (0);
}
It just allocates a 4 MiB of memory and 10 times tries to double its size by reallocation. The simple calls to realloc are interleaved with another allocations of 1 MiB blocks and some random writes in order to minimize the heap allocator "tricks". On the Ubuntu linux machine with 16 GiB RAM, I have the following output:
./realloc_test
initial size == 4194304
initial buffer == 0x7f3604c81010
new size == 8388608
new buffer == 0x7f3604480010
new size == 16777216
new buffer == 0x7f360347f010
new size == 33554432
new buffer == 0x7f360147e010
new size == 67108864
new buffer == 0x7f35fd47d010
new size == 134217728
new buffer == 0x7f35f547c010
new size == 268435456
new buffer == 0x7f35e547b010
new size == 536870912
new buffer == 0x7f35c547a010
new size == 1073741824
new buffer == 0x7f3585479010
new size == 2147483648
new buffer == 0x7f3505478010
new size == 4294967296
new buffer == 0x7f3405477010
Trying to free the buffers.
Successfully freed.
So, all the reallocations up to the 4 GiB seemingly succeed. However, when I set the kernel virtual memory accounting mode to 2 (always check, never overcommit) by this command:
echo 2 > /proc/sys/vm/overcommit_memory
then the output changes to:
./realloc_test
initial size == 4194304
initial buffer == 0x7fade1fa7010
new size == 8388608
new buffer == 0x7fade17a6010
new size == 16777216
new buffer == 0x7fade07a5010
new size == 33554432
new buffer == 0x7fadde7a4010
new size == 67108864
new buffer == 0x7fadda7a3010
new size == 134217728
new buffer == 0x7fadd27a2010
new size == 268435456
new buffer == 0x7fadc27a1010
new size == 536870912
new buffer == 0x7fada27a0010
new size == 1073741824
new buffer == 0x7fad6279f010
new size == 2147483648
tmp_pointer == (nil)
errno == 12
realloc: Cannot allocate memory
The rellocation fails at 2 GiB. The computer's free memory at the time as reported by top has been around 5 GiB, so it is reasonable, because realloc must always allocate continuous block of memory. Now, let's see what happens when running the same program on Mac OS X Lion inside the VirtualBox on the very same machine, but with only 8 GiB of virtual RAM:
./realloc_test
initial size == 4194304
initial buffer == 0x101c00000
new size == 8388608
tmp_pointer == 0x102100000
errno == 22
realloc: Invalid argument
Here, this program has a problem with the very first realloc to 8 MiB. This is in my opinion very strange, because the virtual computer's free memory at the time as reported by top has been around 7 GiB.
The truth is, however, that the realloc in fact succeeded, because its return value is non-NULL (notice the tmp_pointer value just before the program termination). But the very same successful call to realloc has also set the errno to nonzero! Now, what is the correct way to handle this situation?
Should I just ignore the errno and check only the return value from the realloc? But then what about some following errno-based error handlers? This is probably not a good idea.
Should I set errno to zero when realloc returns a non-NULL pointer? This appears to be a solution. But ... I have looked here: http://austingroupbugs.net/view.php?id=374. I don't know how authoritative this resource is, but regarding realloc, it is very clear on this:
"... the standard is also explicit that errno cannot be inspected upon success unless documented, ..."
If I understand correctly, it says that: Yeah, when realloc returns NULL you can look at errno, but not otherwise! This said, am I allowed to reset errno to zero? Without ever looking at it? I find it very unclear to understand and decide what's bad and what's good to do.
I still can't understand why is realloc setting this errno in the first place. And what does its value of "Invalid argument" mean? It is not listed in the man pages, they mention only errno ENOMEM (usually number 12). Could something go wrong? Is something in this simple program causing this behaviour under Mac OS X? Probably yes, ... so, the two main questions are:
I think you are misunderstanding the purpose of errno - it is only defined in the case of a failure. To quote the POSIX standard:
The value of errno should only be examined when it is indicated to be valid by a function's return value. No function in this volume of IEEE Std 1003.1-2001 shall set errno to zero.
(Bold face by me). So you cannot expect errno to be 0 after a successful call. You should only check the return value -- then you can determine why it failed using errno but not the other way around (i.e., the value of errno does not indicate the presence of an error condition - that is not its purpose - in fact, it is not touched if there is no error).