I wanted to ask if it makes sense, and if it does, were there any attempts ?
clarification of the question:
When compiler makers work on the optimizer, they try to make the compiler spit the best assembly sequence according to the target architecture right ?
So in this sense, they must imagine what will be the best instruction to use in case X. So I imagine during the design process, they go by intuition using their assembly knowledge, and some trial/error process with benchmarking of typical cases (of code snippets) they target the optimizer for.
But what if, compiler output choice of assembly code could be decided by a genetic algorithm which would simply try everything with two criterions: of "respecting the client intention as invariant" and "the more speed in the execution test the better".
Using this, it seems possible to prepare the optimizer by feeding the meta-optimizer with thousands of code snippets to optimize, learn the best way to optimize them, and then the final optimizer job (as in "the one delivered in the final compiler") would be to detect which snippet is similar to the client code being parsed and apply the translation.
I hope I'm clear. I'm not suggesting a compiler which would use GO during client's code compilation, but a compiler which would embed a static form of the results found by GO (runned in the labs of the compiler maker).
Yes, it has. An early proponent was Marc Feeley of the Université de Montréal, who used genetic algorithms to find optimal combinations of gcc options for large programs, in particular the Gambit Scheme compiler.
He also coauthored a paper you might find interesting:
Genetic instruction scheduling and register allocation. In International Conference on the Quantitative Evaluation of Systems (QEST'04), pages 76-83, 2004.
You can find a link to a PDF on his research page