When I allocate memory on a specific NUMA node using numa_alloc_onnode() like this :
char *ptr;
if ((ptr = (char *) numa_alloc_onnode(1024,1)) == NULL) {
fprintf(stderr,"Problem in %s line %d allocating memory\n",__FILE__,__LINE__);
return(1);
}
and then use move_pages() to try and confirm that the memory allocated is indeed on node 1 :
printf("ptr is on node %d\n",get_node(ptr));
where
// This function returns the NUMA node that a pointer address resides on.
int get_node(void *p)
{
int status[1];
void *pa;
unsigned long a;
// round p down to the nearest page boundary
a = (unsigned long) p;
a = a - (a % ((unsigned long) getpagesize()));
pa = (void *) a;
if (move_pages(0,1,&pa,NULL,status,0) != 0) {
fprintf(stderr,"Problem in %s line %d calling move_pages()\n",__FILE__,__LINE__);
abort();
}
return(status[0]);
}
I get the answer "ptr is on node -2". From errno-base.h I find that 2 is ENOENT, and the move_pages() man page says that a status of -ENOENT in this context means "The page is not present".
If I replace numa_alloc_onnode() with ordinary malloc() it works fine : I get a node number.
Does anyone have any idea what's going on here?
Thanks in advance.
numa_alloc_onnode(3)
says:
All numa memory allocation policy only takes effect when a
page is actually faulted into the address space of a process
by accessing it. The numa_alloc_* functions take care of this
automatically.
Does this mean you need to store something into the newly-allocated page before the kernel actually gives you the page?