I would like to do simple online-compiler, that take code from browser, send it to server, compile it, and return to browser result of execution.
I am afraid that someone will be execute format C:\ from code or something like it. Is there any way to prevent it?
P.S. I am looking Windows solution.
You'll want to use several layers of operating system features. Run the compiler and resulting program in a firewalled virtual machine as a limited user with restricted access to the operating system.
If all goes well, the OS in the VM will keep them from doing anything like formatting the drive. But if not and they get administrator access to it, the fact that it is a disposable VM with limits placed on it from the outside means the admin access isn't actually worth much. You can simple reimage the VM and start fresh again. (In fact, you might want to image it even when all is normal just as a precaution.)
With the two layers of protection, you should be pretty safe.
Now, in the VM, I'd probably just run Linux anyway. Even if the server is a Windows box, inside the VM you can install a minimal linux distro and use setrlimit and such to clamp down on process memory and cpu time. But if you want Windows too, job objects https://msdn.microsoft.com/en-us/library/ms684161(VS.85).aspx achieve similar resource limits. Just make sure your user account running the compiler and program also have severely restricted permissions on the drive too (on either os).
Of course, you could also do such things on the host OS without the VM... but then if something goes wrong, you can't so easily ditch and replace the whole thing, and administrator access might bypass your other firewall or cpu restrictions. I would definitely use the two layers.
BTW, despite the vm layer, don't forget to still use proper XSRF and XSS care too, just like any other web form.