I'm learning x86 assembly Linux from the book "Programming from the Ground up", and currently I'm learning how to open a file and read from or write to it.
I'm having trouble with the options for opening files. I know 0 is for read-only, 03101 is for write and truncate, but where can I get the full documentation to all the open options?
It's not about WSL, it's about what Linux distro you chose to install in your WSL. Different distros will put things in different places in the filesystem.
locate '*fcntl*.h' is a good way to find the appropriate headers.
You can always compile a C program that includes the documented headers (which will pull in the "real" headers), and look at its gcc -E -dM macro defines. Or even
gcc -E -dM /usr/include/fcntl.h | grep ' O_'
to filter just the O_ macro constants. (That fcntl.h is I think likely to be in the plain /usr/include, not buried somewhere, but maybe that's just my Arch GNU/Linux distro keeping it simple. It keeps Linux-specific libc headers like <asm/unistd.h> in /usr/include/asm/, where you can find unistd_32.h and unistd_64.h for 32 and 64-bit call numbers, respectively.) Or let the usual include-path search happen:
echo '#include <fcntl.h>' | gcc -E -dM - | grep ' O_'
For NASM, beware that some of these constants are written in octal, such as #define O_CREAT 00000100 in asm-generic/fcntl.h. Unlike C and the GNU assembler, in NASM 0100 is the same as decimal 100 (0x64 not 0x40). In NASM, you want 0q100, or equivalently 100o.
So if you just turn the CPP #define text into %define or equ with sed or something, the same text will have a different value when interpreted as a numeric literal by NASM instead of C or GAS.
Or write code that does printf("%x, %x\n", O_CREAT, O_TRUNC) or whatever to print out some constants you're interested in, whatever header they came from. (Or print out their bitwise OR, like O_CREAT|O_TRUNC).
The permission mode bit constants like S_IRUSR are defined in terms of other constants like __S_IREAD so it's a bit of a rats nest to follow; probably just printing it out is a good idea. Or simply write permission bits in octal, like mov edx, 0o666 (NASM) or mov $0666, %edx (GAS). (Letting umask clear the write-for-other bit on file creation).
The names of the constants to look for can be found in the man page, open(2).
See also Looking for mmap flag values for another version of this answer, for those constants.
Trying to create a Makefile file for an assembly/nasm application suggests -imacros to include some specific header files only for their macro definitions, which I guess is an alternative to using -dM to prepare a separate file of macro defs.
Beware that some constants are defined as enum not macros.