PIMAGE_NT_HEADERS ntheaders = (PIMAGE_NT_HEADERS)(PCHAR(virtualpointer) + PIMAGE_DOS_HEADER(virtualpointer)->e_lfanew);
In the code above, virtualpointer points to a memory location that has a PE file loaded.
Why is virtualpointer in brackets in front of PIMAGE_DOS_HEADER?
How does it handle the pointer, and how is e_lfanew getting its value?
I understand the bigger picture that, in the end, ntheaders is getting a memory address that points to the location 0x3c where the NT_HEADER is present, but how is the code working? What's happening behind the scenes?
type(value) is a function-style cast, whereas (type)valu is a C-style cast. But they are both just type-casts nonetheless (for purposes of this code, anyway).
So, in this statement:
PIMAGE_NT_HEADERS ntheaders = (PIMAGE_NT_HEADERS)(PCHAR(virtualpointer) + PIMAGE_DOS_HEADER(virtualpointer)->e_lfanew);
PIMAGE_DOS_HEADER(virtualpointer) is type-casting virtualpointer to an IMAGE_DOS_HEADER* pointer. Let's call this dos.
PCHAR(virtualpointer) is type-casting virtualpointer to a char* pointer. Let's call this pc.
pc + dos->e_lfanew is using pointer arithmetic to advance the value of pc by the number of chars (ie bytes, in this case) specified in e_lfanew, which contains the offset of an IMAGE_NT_HEADERS struct from the start of the PE.
(PIMAGE_NT_HEADERS)(pc + dos->e_lfanew) is type-casting the result of that arithmetic to an IMAGE_NT_HEADERS* pointer.
So, the code is merely taking the starting address stored in virtualpointer, reading the e_lfanew field of the IMAGE_DOS_HEADER located at the front of that memory, advancing forward by the specified number of bytes, and then accessing the IMAGE_NT_HEADERS located at that new location.
You should read An In-Depth Look into the Win32 Portable Executable File Format for more details.