Abstract
The manual generation of heuristics for compiler optimizations is a tedious task which requires both a high amount of expertise and an extensive trial-and-error tuning. The consequence is that many optimizations operate inefficiently. In this chapter, an automatic generation of machine learning based (MLB) heuristics for WCET-aware compiler optimizations is presented. A feasibility study for the generation of WCET-aware heuristics for the source code optimization function inlining is presented. Moreover, this chapter shows that on the one hand MLB WCET-aware heuristics can also be easily constructed at other abstraction levels of the code, namely at assembly level as demonstrated for the optimization loop-invariant code motion. On the other hand, this chapter indicates that an automatic selection of appropriate machine learning algorithms and their parameters can significantly improve the performance of the generated heuristics. To demonstrate the effectiveness of the MLB heuristics, evaluations on real-life benchmarks were performed, showing that the novel heuristics outperform traditional, hand-crafted heuristics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
A.V. Aho, R. Sethi, J.D. Ullman, Compilers: Principles, Techniques, and Tools (Addison-Wesley/Longman, Boston, 1986)
A.W. Appel, Modern Compiler Implementation in C (Cambridge University Press, New York, 1997)
D.F. Bacon, S.L. Graham, O.J. Sharp, Compiler transformations for high-performance computing. ACM Comput. Surv. 26(4), 345–420 (1994)
C.M. Bishop, Pattern Recognition and Machine Learning (Springer, Berlin, 2008)
P. Brazdil, C. Giraud-Carrier, C. Soares, R. Vilalta, Metalearning: Applications to Data Mining (Springer, Berlin, 2009)
L. Breiman, Random forests. Mach. Learn. 45(1), 5–32 (2001)
B. Calder, D. Grunwald, M. Jones et al., Evidence-based static branch prediction using machine learning. ACM Trans. Program. Lang. Syst. 19(1), 188–222 (1997)
J. Cavazos, J.E.B. Moss, Inducing heuristics to decide whether to schedule. SIGPLAN Not. 39(6), 183–194 (2004)
J. Cavazos, M. O’Boyle, Method-specific dynamic compilation using logistic regression, in Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-oriented Programming Systems, Languages, and Applications (OOPSLA), Portland, USA, October 2006, pp. 229–240
J. Cavazos, M. O’Boyle, Automatic tuning of inlining heuristics, in Proceedings of the ACM/IEEE Conference on Supercomputing (SC), Seattle, USA, November 2005, pp. 14–25
K.D. Cooper, L. Torczon, Engineering A Compiler (Morgan Kaufmann, San Francisco, 2004)
K.D. Cooper, M.W. Hall, L. Torczon, An experiment with inline substitution. Softw. Pract. Exp. 21(6), 581–601 (1991)
J.W. Davidson, A.M. Holler, Subprogram inlining: a study of its effects on program execution time. Technical report, University of Virginia, Charlottesville, USA, 1989
M. Guthaus, J. Ringenberg, D. Ernst et al., MiBench: a free, commercially representative embedded benchmark suite, in Proceedings of the 4th IEEE International Workshop on Workload Characteristics (WWC), Austin, USA, December 2001, pp. 3–14
M. Haneda, P. Knijnenburg, H. Wijshoff, Automatic selection of compiler options using non-parametric inferential statistics, in Proceedings of the 14th International Conference on Parallel Architectures and Compilation Techniques (PACT), Saint Louis, USA, September 2005, pp. 123–132
J.H. Holland, Adaptation in Natural and Artificial Systems (MIT Press, Cambridge, 1992)
H. Leather, E. Bonilla, M. O’Boyle, Automatic feature generation for machine learning based optimizing compilation, in Proceedings of the International Symposium on Code Generation and Optimization (CGO), Seattle, USA, March 2009, pp. 81–91
C. Lee, M. Potkonjak, W.H. Mangione-Smith, MediaBench: a tool for evaluating and synthesizing multimedia and communications systems, in Proceedings of the 30th Annual International Symposium on Microarchitecture (MICRO), Research Triangle Park, USA, December 1997, pp. 330–335
P. Lokuciejewski, F. Gedikli, P. Marwedel, K. Morik, Automatic WCET reduction by machine learning based heuristics for function inlining, in Proceedings of the 3rd Workshop on Statistical and Machine Learning Approaches to Architecture and Compilation (SMART), Paphos, Cyprus, January 2009, pp. 1–15
P. Lokuciejewski, M. Stolpe, K. Morik, P. Marwedel, Automatic selection of machine learning models for WCET-aware compiler heuristic generation, in Proceedings of the 4th Workshop on Statistical and Machine Learning Approaches to Architecture and Compilation (SMART), Pisa, Italy, January 2010, pp. 3–17
Mälardalen WCET Research Group. WCET Benchmarks, http://www.mrtc.mdh.se/projects/wcet, March 2010
A. McGovern, J.E.B. Moss, Scheduling straight-line code using reinforcement learning and rollouts, in Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS), Denver, USA, November 1999, pp. 903–909
G. Memik, W.H. Mangione-Smith, W. Hu, NetBench: a benchmarking suite for network processors, in Proceedings of the International Conference on Computer-Aided Design (ICCAD), San Jose, USA, November 2001, pp. 39–42
I. Mierswa, Non-convex and multi-objective optimization in data mining, PhD thesis, TU Dortmund University, 2008
I. Mierswa, M. Wurst, R. Klinkenberg et al., YALE: rapid prototyping for complex data mining tasks, in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), Philadelphia, USA, August 2006, pp. 935–940
A. Monsifrot, F. Bodin, R. Quiniou, A machine learning approach to automatic production of compiler heuristics, in Proceedings of the 10th International Conference on Artificial Intelligence: Methodology, Systems, and Applications (AIMSA), Varna, Bulgaria, September 2002, pp. 41–50
S.S. Muchnick, Advanced Compiler Design and Implementation (Morgan Kaufmann, San Francisco, 1997)
T. Russell, A.M. Malik, M. Chase et al., Learning heuristics for the superblock instruction scheduling problem. IEEE Trans. Knowl. Data Eng. 21(10), 1489–1502 (2009)
Y.N. Srikant, P. Shankar, The Compiler Design Handbook: Optimizations and Machine Code Generation, 2nd edn. (CRC Press, Boca Raton, 2007)
M. Stephenson, S. Amarasinghe, Predicting unroll factors using supervised classification, in Proceedings of the International Symposium on Code Generation and Optimization (CGO), San Jose, USA, March 2005, pp. 123–134
M. Stephenson, S. Amarasinghe, M. Martin, U.M. O’Reilly, Meta optimization: improving compiler heuristics with machine learning. SIGPLAN Not. 38(5), 77–90 (2003)
UTDSP Benchmark Suite. http://www.eecg.toronto.edu/~corinna/DSP/infrastructure/UTDSP.html, March 2010
K. Vaswani, M.J. Thazhuthaveetil, Y. Srikant et al., Microarchitecture sensitive empirical models for compiler optimizations, in Proceedings of the International Symposium on Code Generation and Optimization (CGO), San Jose, USA, March 2007, pp. 131–143
V. Zivojnović, J. MartÃnez Velarde, C. Schläger et al., DSPstone: a DSP-oriented benchmarking methodology, in Proceedings of the International Conference on Signal Processing and Technology (ICSPAT), Dallas, USA, January 1994, pp. 715–720
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media B.V.
About this chapter
Cite this chapter
Lokuciejewski, P., Marwedel, P. (2011). Machine Learning Techniques in Compiler Design. In: Worst-Case Execution Time Aware Compilation Techniques for Real-Time Systems. Embedded Systems. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-9929-7_6
Download citation
DOI: https://doi.org/10.1007/978-90-481-9929-7_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-90-481-9928-0
Online ISBN: 978-90-481-9929-7
eBook Packages: EngineeringEngineering (R0)