Is it possible to use segmented memory in x86 when you have a program running in user-mode? If so, are there any tools (e.g. C/C++ compilers) to facilitate this? It seems to be a really powerful mechanism, allowing the running of programs within programs.
I've heard that Chromium uses segmentation for sandboxing in the Native Client, but I'm not sure how, or if it uses assembly or built-in compiler support.
Native Client uses both compiler support (of the asm
extension, a gnu form here) and assembly for working with segment registers:
http://www.google.com/codesearch/p?hl=en#IAaH75l62fw/service_runtime/linux/sel_segments.c&l=71
uint16_t NaClGetEs(void)
{
uint16_t seg1;
asm("mov %%es, %0" : "=r" (seg1) : );
return seg1;
}
void NaClSetEs(uint16_t seg1)
{
asm("movw %0, %%es;" : : "r" (seg1));
}
So, it is possible, but rather hard to program.
There is a paper about Native Client: http://nativeclient.googlecode.com/svn/trunk/src/native_client/documentation/nacl_paper.pdf
NaCl does additional code verification (it disallows untrusted code to reprogram segment registers), but changes the segments itself. This lead to some problems, .... read the section 3 of the paper.
Another user of segmentation is OpenBSD OS with W^X technology on i386 (CS segment limiting) http://www.openbsd.org/papers/ven05-deraadt/mgp00010.html (whole presentation is http://www.openbsd.org/papers/ven05-deraadt/index.html )