ASK: Adaptive Sampling Kit for Performance Characterization

  • Pablo de Oliveira Castro
  • Eric Petit
  • Jean Christophe Beyler
  • William Jalby
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7484)


Characterizing performance is essential to optimize programs and architectures. The open source Adaptive Sampling Kit (ASK) measures the performance trade-offs in large design spaces. Exhaustively sampling all points is computationally intractable. Therefore, ASK concentrates exploration in the most irregular regions of the design space through multiple adaptive sampling methods. The paper presents the ASK architecture and a set of adaptive sampling strategies, including a new approach: Hierarchical Variance Sampling. ASK’s usage is demonstrated on two performance characterization problems: memory stride accesses and stencil codes. ASK builds precise models of performance with a small number of measures. It considerably reduces the cost of performance exploration. For instance, the stencil code design space, which has more than 31.108 points, is accurately predicted using only 1 500 points.


Root Mean Square Error Design Space Adaptive Sampling Performance Characterization Irregular Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Simpson, T., Lin, D., Chen, W.: Sampling strategies for computer experiments: design and analysis. International Journal of Reliability and Applications 2(3), 209–240 (2001)Google Scholar
  2. 2.
    Stein, M.: Large sample properties of simulations using Latin hypercube sampling. Technometrics, 143–151 (1987)Google Scholar
  3. 3.
    Johnson, M., Moore, L., Ylvisaker, D.: Minimax and maximin distance designs. Journal of Statistical Planning and Inference 26(2), 131–148 (1990)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Diwekar, U., Kalagnanam, J.: Efficient sampling technique for optimization under uncertainty. AIChE Journal 43(2), 440–447 (1997)CrossRefGoogle Scholar
  5. 5.
    Li, B., Peng, L., Ramadass, B.: Accurate and efficient processor performance prediction via regression tree based modeling. Journal of Systems Architecture 55(10-12), 457–467 (2009)CrossRefGoogle Scholar
  6. 6.
    Ridgeway, G.: Generalized Boosted Models: A guide to the gbm package. Update 1, 1 (2007)Google Scholar
  7. 7.
    Gramacy, R., Lee, H.: Adaptive design and analysis of supercomputer experiments. Technometrics 51(2), 130–145 (2009)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Gramacy, R.: tgp: An R package for Bayesian nonstationary, semiparametric nonlinear regression and design by treed gaussian process models. Journal of Statistical Software 19(9), 6 (2007)Google Scholar
  9. 9.
    Cohn, D.: Neural network exploration using optimal experiment design. Neural Networks 9(6), 1071–1083 (1996)CrossRefGoogle Scholar
  10. 10.
    Settles, B.: Active Learning Literature Survey. Science 10(3), 237–304 (1995)Google Scholar
  11. 11.
    Gorissen, D., Couckuyt, I., Demeester, P., Dhaene, T., Crombecq, K.: A surrogate modeling and adaptive sampling toolbox for computer based design. The Journal of Machine Learning Research 11, 2051–2055 (2010)Google Scholar
  12. 12.
    Crombecq, K., Gorissen, D., Deschrijver, D., Dhaene, T.: A Novel Hybrid Sequential Design Strategy for Global Surrogate Modeling of Computer Experiments. SIAM Journal on Scientific Computing 33, 1948 (2011)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Dasgupta, S., Hsu, D.: Hierarchical sampling for active learning. In: Proceedings of the 25th International Conference on Machine learning, pp. 208–215. ACM (2008)Google Scholar
  14. 14.
    Breiman, L., Friedman, J., Olshen, R., Stone, C., Steinberg, D., Colla, P.: CART: Classification and regression trees. Wadsworth, Belmont (1983)Google Scholar
  15. 15.
    Atkinson, E., Therneau, T.: An introduction to recursive partitioning using the RPART routines. Mayo Foundation, Rochester (2000)Google Scholar
  16. 16.
    McKay, A.: Distribution of the Coefficient of Variation and the Extended t Distribution. Journal of the Royal Statistical Society 95(4), 695–698 (1932)CrossRefGoogle Scholar
  17. 17.
    Friedman, J.: Greedy function approximation: a gradient boosting machine. Annals of Statistics, 1189–1232 (2001)Google Scholar
  18. 18.
    Datta, K., Murphy, M., Volkov, V., Williams, S., Carter, J., Oliker, L., Patterson, D., Shalf, J., Yelick, K.: Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures. In: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, pp. 1–12. IEEE Press (2008)Google Scholar
  19. 19.
    Treibig, J., Wellein, G., Hager, G.: Efficient multicore-aware parallelization strategies for iterative stencil computations. J. Comput. Science 2(2), 130–137 (2011)CrossRefGoogle Scholar
  20. 20.
    Dursun, H., Nomura, K.-i., Peng, L., Seymour, R., Wang, W., Kalia, R.K., Nakano, A., Vashishta, P.: A Multilevel Parallelization Framework for High-Order Stencil Computations. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 642–653. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  21. 21.
    Maron, O., Moore, A.: Hoeffding races: Accelerating model selection search for classification and function approximation. Robotics Institute, 263 (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Pablo de Oliveira Castro
    • 1
  • Eric Petit
    • 2
  • Jean Christophe Beyler
    • 3
  • William Jalby
    • 1
  1. 1.Exascale Computing ResearchUniversity of Versailles - UVSQFrance
  2. 2.LRC ITACAUniversity of Versailles - UVSQFrance
  3. 3.Intel CorporationUSA

Personalised recommendations