I've been looking recently into creating a new native language. I understand the (very) basics of the PE format and I've grabbed an assembler with a fairly kind interface off the webs, which I've successfully used to implement some simple functions. But I've run into a problem using functions from a library. The only way that I've called library functions from a dynamically compiled function previously is to pass in the function pointer manually- something I can't do if I create PE files and execute them in their own process. Now, I'm not planning on using the CRT, but I will need access to the Win API to implement my own standard libraries. How do I generate a reference to a WinAPI function so that the PE loader will patch it up?
You need to write an import table. It's basically a list of function names that you wish to use in your application. It's pointed to by the PE header. The loader loads the DLL files into the process memory space for you, finds the requested function in their export table and leaves the address for it in the import table. You then usually dereference that and jmp there.
Check out Izelion's assembly tutorial for the full details and for asm examples.