numaif.h: MPOL_LOCAL undeclared for use in mbind

Question

According to the mbind man page, one possible mode is MPOL_LOCAL, which places the memory region in the same node of the CPU that triggered the allocation:

#include <numaif.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>    

#define N 134217728

int main() {
    uint64_t *a = (uint64_t*) malloc(N*sizeof(uint64_t));
    mbind(a, N, MPOL_LOCAL, 0, 0, MPOL_MF_STRICT | MPOL_MF_MOVE);
    printf("Hello world!\n");
    return 0;
}

However, the symbol is simply not defined.

$ gcc-8 -lnuma example.c
example.c: In function ‘main’:
example.c:10:14: error: ‘MPOL_LOCAL’ undeclared (first use in this function); did you mean ‘MPOL_MAX’?
  mbind(a, N, MPOL_LOCAL, 0, 0, MPOL_MF_STRICT | MPOL_MF_MOVE);
              ^~~~~~~~~~
              MPOL_MAX
example.c:10:14: note: each undeclared identifier is reported only once for each function it appears in

Changing to e.g. MPOL_INTERLEAVE makes it compile and display Hello world! just fine.

What's going on here? At this stage I'm 100% puzzled.

I've tried with gcc/g++ 4.9.2, 5 and 8; in three different machines running the kernels 4.17.12+ (no idea where it came from), 4.18.10 (compiled myself) and 4.15.0 (included in latest Linux Mint). libnuma-dev is up to date.

Answer 1

MPOL_LOCAL is declared kernel side in uapi/linux/mempolicy.h and is equal 4. It's in UAPI, so actually you can #include <linux/mempolicy.h>.

MPOL_LOCAL should be declared user side in numaif.h as in the man page. I fail to see why it isn't, alongside with other defines. Also the definition for MPOL_MAX changes - kernel side it is equal to MPOL_LOCAL + 1 = 5, but in numaif.h it's equal to MPOL_INTERLEAVE = 3. I would expect MPOL_MAX to be either one greater then the maximum MPOL or equal maximum value on both sides, but kernel and user space tool define it differently.

I think it deserves to post an issue to numactl to notify the developers. According to man page it should defined. I don't know why it isn't.

As for now, I would just

#include <numaif.h>
#ifndef MPOL_LOCAL
#define MPOL_LOCAL 4
#endif

I have browsed the web to find out how different programs handle this. I think programs just define all MPOL_* symbols themselves, see hwloc or stress-ng. fio just defined MPOL_LOCAL themself to be equal 4. Open-mpi even marks that numaif.h does not define MPOL_LOCAL and they need to define it, see here.