A Bayesian Method of Online Automatic Tuning
This chapter discusses mathematical methods for online automatic tuning. After formulating the abstract model of automatic tuning, we review the proposed method, which comprises several novel concepts of automatic tuning, such as online automatic tuning, Bayesian data analysis for quantitative treatments of uncertainties, Bayesian suboptimal sequential experimental design, asymptotic optimality, finite startup, and infinite dilution. Experimental results reveal that the Bayesian sequential experimental design has advantages over random sampling, although random sampling combined with an accurate cost function model can be as good as the Bayesian sequential experimental design.
KeywordsObjective Function Cost Function Posterior Distribution Prior Distribution Tuning Parameter
The author sincerely appreciates the members of the Automatic Tuning Research Group for engaging in valuable discussions and collaborations and providing valuable suggestions. The author is also grateful to Prof. Akimichi Takemura, Prof. Tatsuya Kubokawa, Dr. Kazuki Yoshizoe, and Mr. Junya Honda for their invaluable and essential suggestions.
This study is supported in part by a Grant-in-Aid for Scientific Research “Research on Mathematical Core for Robust Auto-Tuning System in Information Explosion Era” from MEXT Japan and the Core Research of Evolutional Science and Technology (CREST) project “ULP-HPC: Ultra Low-Power, High-Performance Computing via Modeling and Optimization of Next Generation HPC Technologies” of the Japan Science and Technology Agency (JST).
- 1.Whaley RC, Dongarra JJ (1998) Automatically tuned linear algebra software. In: Proceedings of SC98 (CD-ROM)Google Scholar
- 2.Frigo M, Johnson SG (1998) FFTW: an adaptive software architecture for the FFT. In: Proceedings of ICASSP ’98, vol 3, pp 1381–1384Google Scholar
- 4.Imamura T (2007) Recursive multi-factoring algorithm for MPI allreduce. In: Proc. IASTED int’l conf. parallel and distributed computing and networks (PDCN 2007), pp (551)135–145Google Scholar
- 5.Katagiri T, Voemel C, Demmel J (2007) Automatic performance tuning for the multi-section with multiple eigenvalues method for the symmetric eigenproblem. In: Selected papers of PARA’06, Lecture Notes in Computer Science, vol 4699. Springer, Berlin, pp 938–948Google Scholar
- 6.Naono K, Sakurai T, Egi M (2008) Research trends on automatic tuning methods for matrix computations and proposal of a new run-time automatic tuning method. In: Int’l workshop on par. mat. alg. appl. (PMAA08)Google Scholar
- 7.Fukaya T, Yamamoto Y, Zhang S-L (2008) A dynamic programming approach to optimizing the blocking strategy for the householder QR decomposition. In: Proceedings of IEEE international conference on cluster computing 2008 (Proc. int’l workshop on automatic performance tuning (iWAPT2008)), pp 402–410Google Scholar
- 10.Eijkhout V (2006) A self-adapting system for linear solver selection. In: Proc. 1st int’l workshop on automatic performance tuning (iWAPT2006), pp 44–53Google Scholar
- 11.Suda R (2007) A Bayesian method for online code selection: toward efficient and robust methods of automatic tuning. In: Proc. 2nd int’l workshop on automatic performance tuning (iWAPT2007), pp 23–32Google Scholar
- 12.Suda R (2008) A Bayesian approach to automatic performance tuning. In: 13th SIAM conf. para. proc. sci. comp. (PP08) (oral presentation)Google Scholar
- 18.Vermorel J (2005) Mohri M Multi-armed bandit algorithms and empirical evaluation. In: Proc. Euro. conf. machine learning (ECML 2005), Lecture Notes in Computer Science, vol 3720. Springer, Berlin, pp 437–448Google Scholar