Abstract
Characterizing performance is essential to optimize programs and architectures. The open source Adaptive Sampling Kit (ASK) measures the performance trade-offs in large design spaces. Exhaustively sampling all points is computationally intractable. Therefore, ASK concentrates exploration in the most irregular regions of the design space through multiple adaptive sampling methods. The paper presents the ASK architecture and a set of adaptive sampling strategies, including a new approach: Hierarchical Variance Sampling. ASK’s usage is demonstrated on two performance characterization problems: memory stride accesses and stencil codes. ASK builds precise models of performance with a small number of measures. It considerably reduces the cost of performance exploration. For instance, the stencil code design space, which has more than 31.108 points, is accurately predicted using only 1 500 points.
Chapter PDF
References
Simpson, T., Lin, D., Chen, W.: Sampling strategies for computer experiments: design and analysis. International Journal of Reliability and Applications 2(3), 209–240 (2001)
Stein, M.: Large sample properties of simulations using Latin hypercube sampling. Technometrics, 143–151 (1987)
Johnson, M., Moore, L., Ylvisaker, D.: Minimax and maximin distance designs. Journal of Statistical Planning and Inference 26(2), 131–148 (1990)
Diwekar, U., Kalagnanam, J.: Efficient sampling technique for optimization under uncertainty. AIChE Journal 43(2), 440–447 (1997)
Li, B., Peng, L., Ramadass, B.: Accurate and efficient processor performance prediction via regression tree based modeling. Journal of Systems Architecture 55(10-12), 457–467 (2009)
Ridgeway, G.: Generalized Boosted Models: A guide to the gbm package. Update 1, 1 (2007)
Gramacy, R., Lee, H.: Adaptive design and analysis of supercomputer experiments. Technometrics 51(2), 130–145 (2009)
Gramacy, R.: tgp: An R package for Bayesian nonstationary, semiparametric nonlinear regression and design by treed gaussian process models. Journal of Statistical Software 19(9), 6 (2007)
Cohn, D.: Neural network exploration using optimal experiment design. Neural Networks 9(6), 1071–1083 (1996)
Settles, B.: Active Learning Literature Survey. Science 10(3), 237–304 (1995)
Gorissen, D., Couckuyt, I., Demeester, P., Dhaene, T., Crombecq, K.: A surrogate modeling and adaptive sampling toolbox for computer based design. The Journal of Machine Learning Research 11, 2051–2055 (2010)
Crombecq, K., Gorissen, D., Deschrijver, D., Dhaene, T.: A Novel Hybrid Sequential Design Strategy for Global Surrogate Modeling of Computer Experiments. SIAM Journal on Scientific Computing 33, 1948 (2011)
Dasgupta, S., Hsu, D.: Hierarchical sampling for active learning. In: Proceedings of the 25th International Conference on Machine learning, pp. 208–215. ACM (2008)
Breiman, L., Friedman, J., Olshen, R., Stone, C., Steinberg, D., Colla, P.: CART: Classification and regression trees. Wadsworth, Belmont (1983)
Atkinson, E., Therneau, T.: An introduction to recursive partitioning using the RPART routines. Mayo Foundation, Rochester (2000)
McKay, A.: Distribution of the Coefficient of Variation and the Extended t Distribution. Journal of the Royal Statistical Society 95(4), 695–698 (1932)
Friedman, J.: Greedy function approximation: a gradient boosting machine. Annals of Statistics, 1189–1232 (2001)
Datta, K., Murphy, M., Volkov, V., Williams, S., Carter, J., Oliker, L., Patterson, D., Shalf, J., Yelick, K.: Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, pp. 1–12. IEEE Press (2008)
Treibig, J., Wellein, G., Hager, G.: Efficient multicore-aware parallelization strategies for iterative stencil computations. J. Comput. Science 2(2), 130–137 (2011)
Dursun, H., Nomura, K.-i., Peng, L., Seymour, R., Wang, W., Kalia, R.K., Nakano, A., Vashishta, P.: A Multilevel Parallelization Framework for High-Order Stencil Computations. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 642–653. Springer, Heidelberg (2009)
Maron, O., Moore, A.: Hoeffding races: Accelerating model selection search for classification and function approximation. Robotics Institute, 263 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
de Oliveira Castro, P., Petit, E., Beyler, J.C., Jalby, W. (2012). ASK: Adaptive Sampling Kit for Performance Characterization. In: Kaklamanis, C., Papatheodorou, T., Spirakis, P.G. (eds) Euro-Par 2012 Parallel Processing. Euro-Par 2012. Lecture Notes in Computer Science, vol 7484. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32820-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-32820-6_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32819-0
Online ISBN: 978-3-642-32820-6
eBook Packages: Computer ScienceComputer Science (R0)