Search code examples
optimizationcompiler-constructioncompiler-optimizationjit

What can a JIT compiler do that an AOT compiler cannot?

A Just-in-Time (JIT) compiler can optimize a program based on runtime information that is unavailable to an Ahead-of-Time (AOT) compiler.

The most obvious example of this runtime information is the target platform, e.g. the exact CPU on which the program is running, or any accelerators such as GPUs that might be available. This is the sense in which OpenCL is JIT-compiled.

But suppose we do know ahead of time what the target platform is: we know which SIMD extensions will be available, etc. What other runtime information can a JIT-compiler exploit that is unavailable to an AOT-compiler?

A HotSpot-style JIT-compiler will automatically optimize a program's hot spots... but can't an AOT-compiler just optimize the whole program, hot spots and all?

I would like some examples of specific optimizations that a JIT-compiler can perform which an AOT-compiler cannot. Bonus points if you can provide any evidence for the effectiveness of such optimizations in "real world" scenarios.

Solution

  • A JIT compiler can optimize based on run-time information which result in stricter border conditions which were not provable at compile time. Examples:

    • It can see that a memory location is not aliased (because the code path taken never aliased it) and thus keep the variable in a register;
    • it can eliminate a test for a condition which can never occur (e.g. based on the current values of parameters);
    • it has access to the complete program and can inline code where it sees fit;
    • it can perform branch prediction based on the specific use pattern at run time so that it's optimal.

    The inlining is principally also open to link time optimization of modern compilers/linkers but may lead to prohibitive code bloat if applied throughout the code just in case; at run time it can be applied just where necessary.

    The branch prediction can be improved with normal compilers if the program is compiled twice, with a test run inbetween; in a first run the code is instrumented so that it generates profiling data which is used in the production compilation run to optimize branch prediction. The prediction is also less than optimal if the test run was not typical (and it's not always easy to prduce typical test data, or the usage patterns may shift over the life time of the program) .

    Additionally, both link time and run time data optimization with static compilation need significant effort in the build process (to a degree that I have not seen them employed in production in the 10 or so places where I have worked in my life); with a JIT they are on by default.