According to the information I could gather on .NET and Java execution environment, the current state of affairs is follows:
Modern Java VM are capable of performing continuous recompilation, which combined with profiling can yield great performance improvements. Older JVMs employed JIT. More information in this article: http://www.ibm.com/developerworks/library/j-jtp12214/ and especially: Java theory and practice: Dynamic compilation and performance measurement
.NET uses JIT or NGEN to generate native code, but once the native code is generated, no further (runtime) optimizations are performed.
Benchmarks aside and with no intention to escalate holy wars, does this mean that Java Hotspot VM is one generation ahead of .Net. Will these technologies employed at Java VM eventually find its way into .NET runtime?
I've never benchmarked the two to compare, and I'm more familiar with the Sun JVM, I can only speak in general terms about JITs.
There are always tradeoffs with optimizations, and not all optimizations work all the time. However, here are some modern JIT techniques. I think this can be the beginning of a good conversation if we stick to the technical stuff:
There's also features that are helpful as far as good implementations of a VM go:
Based on these features and many more, we can compare VMs, and not just "Java" versus ".NET" but, say, Sun's JVM versus IBM's JVM versus .NET versus Mono.
For example, Sun's JVM doesn't do tail-call optimization, IIRC, but IBM's does.