Efficient Program Compilation Through Machine Learning Techniques

  • Gennady Pekhimenko
  • Angela Demke Brown


The wealth of available compiler optimizations leads to the dual problems of finding the best set of optimizations and the best heuristic parameters to tune each optimization. We describe how machine learning techniques, such as logistic regression, can be used to address these problems. We focus on decreasing the compile time for a static commercial compiler, while preserving the execution time. We show that we can speed up the compile process by at least a factor of two with almost the same generated code quality on the SPEC2000 benchmark suite, and that our logistic classifier achieves the same prediction quality for non-SPEC benchmarks.


Execution Time Training Data Feature Vector Compiler Optimization Reduce Execution Time 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Agakov F, Bonilla E, Cavazos J, et al. (2006) Using machine learning to focus iterative optimization. In Proceedings of the 4th international symposium on code generation and optimization (CGO’06), March 2006, pp 295–305Google Scholar
  2. 2.
    Calder B, Grunwald D, Jones M, Lindsay D, Martin J, Mozer M, Zorn B (1997) Evidence-based static branch prediction using machine learning. ACM Trans Program Lang Syst 19(1):188–222CrossRefGoogle Scholar
  3. 3.
    Cavazos J, Fursin G, Agakov F et al. (2007) Rapidly selecting good compiler optimizations using performance counters. In Proceedings of the 2007 international symposium on code generation and optimization (CGO ’07), March 2007, pp 185–197Google Scholar
  4. 4.
    Cavazos J, O’Boyle MFP (2006) Method-specific dynamic compilation using logistic regression. In Proceedings of OOPSLA ’06, October 2006, pp 229–240Google Scholar
  5. 5.
    Cooper KD, Subramanian D, Torczon L (2002) Adaptive optimizing compilers for the 21st century. J Supercomput 23(1):7–22MATHCrossRefGoogle Scholar
  6. 6.
    Fursin G, Miranda C, Temam O, Namolaru M, Yom-Tov E, Zaks A, Mendelson B, Barnard P, Ashton E, Courtois E, Bodin F, Bonilla E, Thomson J, Leather H, Williams C, O’Boyle M (2008) MILEPOST GCC: Machine-learning-based research compiler. In Proceedings of the GCC Developers’ Summit, June 2008Google Scholar
  7. 7.
    ORC (2008) Open Research Compiler for Itanium Processor Family.
  8. 8.
    PathScale (2004) EKOPath compilers.
  9. 9.
    Pan Z, Eigenmann R (2006) Fast, automatic, procedure-level performance tuning. In Proceedings of the 15th International Conference on Parallel Architecture and Compilation Techniques (PACT’06), September 2006, pp 173–181Google Scholar
  10. 10.
    Pekhimenko G (2008) Machine learning algorithms for choosing compiler heuristics. Master’s thesis, University of Toronto
  11. 11.
    Stephenson M, Amarasinghe S (2005) Predicting unroll factors using supervised classification. In Proceedings of the 2005 international symposium on code generation and optimization (CGO’05), March 2005, pp 123–134Google Scholar
  12. 12.
    Stephenson M, Amarasinghe S, Martin M, O’Reilly UM (2003) Meta optimization: improving compiler heuristics with machine learning. In Proceedings of the 2003 ACM SIGPLAN Conference on Programing Language Design and Implimentation (PLDI ’03), June 2003, pp 77–90Google Scholar
  13. 13.
    Seymour K, You H, Dongarra J (2008) A comparison of search heuristics for empirical code optimization. In Proceedings of the 2008 IEEE international conference on cluster computing (3rd Intl Wkshp on Automatic Perf. Tuning), pp 421–429Google Scholar
  14. 14.
    Standard Performance Evaluation Corporation (2000) SPEC CPU2000 benchmarks.
  15. 15.
    Tal A (2007) Method and system for managing heuristic properties. US Patent Application No. 20070089104, 19 April 2007Google Scholar

Copyright information

© Springer New York 2011

Authors and Affiliations

  1. 1.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations