Advertisement

Can Linear Approximation Improve Performance Prediction ?

  • Vlastimil Babka
  • Petr Tůma
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6977)

Abstract

Software performance evaluation relies on the ability of simple models to predict the performance of complex systems. Often, however, the models are not capturing potentially relevant effects in system behavior, such as sharing of memory caches or sharing of cores by hardware threads. The goal of this paper is to investigate whether and to what degree a simple linear adjustment of service demands in software performance models captures these effects and thus improves accuracy. Outlined experiments explore the limits of the approach on two hardware platforms that include shared caches and hardware threads, with results indicating that the approach can improve throughput prediction accuracy significantly, but can also lead to loss of accuracy when the performance models are otherwise defective.

Keywords

Performance modeling resource sharing linear models 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Babka, V., Bulej, L., Decky, M., Kraft, J., Libic, P., Marek, L., Seceleanu, C., Tuma, P.: Resource Usage Modeling, Q-ImPrESS Project Deliverable D3.3 (2008), http://www.q-impress.eu/
  2. 2.
    Babka, V.: Cache Sharing Sensitivity of SPEC CPU 2006 Benchmarks, Tech. Rep. No. 2009/3, Dep. of SW Engineering, Charles University in Prague (June 2009), http://d3s.mff.cuni.cz/
  3. 3.
    Babka, V., Bulej, L., Ciancone, A., Filieri, A., Hauck, M., Libic, P., Marek, L., Stammel, J., Tuma, P.: Prediction Validation, Q-ImPrESS Project Deliverable D4.2 (2010), http://www.q-impress.eu/
  4. 4.
    Babka, V., Libič, P., Tůma, P.: Timing Penalties Associated with Cache Sharing. In: Proceedings of MASCOTS 2009. IEEE, Los Alamitos (2009)Google Scholar
  5. 5.
    Babka, V., Marek, L., Tůma, P.: When Misses Differ: Investigating Impact of Cache Misses on Observed Performance. In: Proceedings of ICPADS 2009, pp. 112–119. IEEE, Los Alamitos (2009)Google Scholar
  6. 6.
    Babka, V., Tůma, P.: Investigating Cache Parameters of x86 Family Processors. In: Kaeli, D., Sachs, K. (eds.) SPEC Benchmark Workshop 2009. LNCS, vol. 5419, pp. 77–96. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Babka, V., Tůma, P., Bulej, L.: Validating Model-Driven Performance Predictions on Random Software Systems. In: Heineman, G.T., Kofron, J., Plasil, F. (eds.) QoSA 2010. LNCS, vol. 6093, pp. 3–19. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Becker, S., Koziolek, H., Reussner, R.: The Palladio Component Model for Model-driven Performance Prediction. J. Syst. Softw. 82(1) (2009)Google Scholar
  9. 9.
    Blackburn, S.M., Cheng, P., McKinley, K.S.: Myths and Realities: The Performance Impact of Garbage Collection. SIGMETRICS Perform. Eval. Rev. 32(1) (2004)Google Scholar
  10. 10.
    Chandra, D., Guo, F., Kim, S., Solihin, Y.: Predicting Inter-Thread Cache Contention on a Chip Multi-Processor Architecture. In: Proceedings of HPCA 2005. IEEE CS, Los Alamitos (2005)Google Scholar
  11. 11.
    Click, C.: Evaluate 2010 Keynote (October 2010), http://evaluate2010.inf.usi.ch/
  12. 12.
    Grassi, V., Mirandola, R., Randazzo, E., Sabetta, A.: KLAPER: An Intermediate Language for Model-Driven Predictive Analysis of Performance and Reliability. In: Rausch, A., Reussner, R., Mirandola, R., Plášil, F. (eds.) The Common Component Modeling Example. LNCS, vol. 5153, pp. 327–356. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Happe, J., Westermann, D., Sachs, K., Kapová, L.: Statistical Inference of Software Performance Models for Parametric Performance Completions. In: Heineman, G.T., Kofron, J., Plasil, F. (eds.) QoSA 2010. LNCS, vol. 6093, pp. 20–35. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Kalibera, T., Bulej, L., Tůma, P.: Benchmark Precision and Random Initial State. In: Proceedings of SPECTS 2005, pp. 853–862. SCS (June 2005)Google Scholar
  15. 15.
    Kounev, S.: Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri Nets. IEEE Trans. Software Eng. 32(7) (2006)Google Scholar
  16. 16.
    Kounev, S., Buchmann, A.: SimQPN: A Tool and Methodology for Analyzing Queueing Petri Net Models by Means of Simulation. Perform. Eval. 63(4) (2006)Google Scholar
  17. 17.
    Lavenberg, S.S., Squillante, M.S.: Performance Evaluation in Industry: A Personal Perspective. In: Reiser, M., Haring, G., Lindemann, C. (eds.) Dagstuhl Seminar 1997. LNCS, vol. 1769, pp. 3–13. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  18. 18.
    Libič, P., Tůma, P.: Towards Garbage Collection Modeling, Tech. Rep. No. 20011/1, Dep. of Distributed and Dependable Systems, Charles University in Prague (January 2011), http://d3s.mff.cuni.cz/
  19. 19.
    Libič, P., Tůma, P., Bulej, L.: Issues in Performance Modeling of Applications with Garbage Collection. In: Proceedings of QUASOSS 2009, pp. 3–10. ACM, New York (2009)Google Scholar
  20. 20.
    Liu, F., Guo, F., Solihin, Y., Kim, S., Eker, A.: Characterizing and Modeling the Behavior of Context Switch Misses. In: Proceedings of PACT 2008. ACM, New York (2008)Google Scholar
  21. 21.
    Mytkowicz, T., Diwan, A., Hauswirth, M., Sweeney, P.F.: Producing Wrong Data Without Doing Anything Obviously Wrong! In: Proceedings of ASPLOS 2009, pp. 265–276. ACM, New York (2009)Google Scholar
  22. 22.
    Standard Performance Evaluation Corporation: SPEC CPU 2006 Benchmark, http://www.spec.org/cpu2006/
  23. 23.
    The Q-ImPrESS Project Consortium: Quality Impact Prediction for Evolving Service-oriented Software, http://www.q-impress.eu/
  24. 24.
    Wasserman, L.: All of Statistics: A Concise Course in Statistical Inference. Springer, Heidelberg (2004)CrossRefzbMATHGoogle Scholar
  25. 25.
    Xu, C., Chen, X., Dick, R.P., Mao, Z.M.: Cache Contention and Application Performance Prediction for Multi-core Systems. In: Proceedings of ISPASS 2010. IEEE, Los Alamitos (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Vlastimil Babka
    • 1
  • Petr Tůma
    • 1
  1. 1.Department of Distributed and Dependable SystemsCharles University in Prague, Faculty of Mathematics and PhysicsPrague 1Czech Republic

Personalised recommendations