Skip to main content
SpringerLink
Log in

Validating Model-Driven Performance Predictions on Random Software Systems

  • Conference paper
Research into Practice – Reality and Gaps (QoSA 2010)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 6093))

Included in the following conference series:

Abstract

Software performance prediction methods are typically validated by taking an appropriate software system, performing both performance predictions and performance measurements for that system, and comparing the results. The validation includes manual actions, which makes it feasible only for a small number of systems.

To significantly increase the number of systems on which software performance prediction methods can be validated, and thus improve the validation, we propose an approach where the systems are generated together with their models and the validation runs without manual intervention. The approach is described in detail and initial results demonstrating both its benefits and its issues are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Avritzer, A., Weyuker, E.J.: The Automatic Generation of Load Test Suites and the Assessment of the Resulting Software. IEEE Trans. Software Eng. 21(9) (1995)

    Google Scholar 

  2. Babka, V., Bulej, L., Decky, M., Kraft, J., Libic, P., Marek, L., Seceleanu, C., Tuma, P.: Resource Usage Modeling, Q-ImPrESS Project Deliverable D3.3 (2008), http://www.q-impress.eu/

  3. Balsamo, S., Di Marco, A., Inverardi, P., Simeoni, M.: Model-Based Performance Prediction in Software Development: A Survey. IEEE Trans. Software Eng. 30(5) (2004)

    Google Scholar 

  4. Bause, F.: Queueing Petri Nets - A Formalism for the Combined Qualitative and Quantitative Analysis of Systems. In: Proc. 5th Intl. W. on Petri Nets and Performance Models. IEEE CS, Los Alamitos (1993)

    Google Scholar 

  5. Becker, S., Bulej, L., Bures, T., Hnetynka, P., Kapova, L., Kofron, J., Koziolek, H., Kraft, J., Mirandola, R., Stammel, J., Tamburrelli, G., Trifu, M.: Service Architecture Meta Model, Q-ImPrESS Deliverable D2.1 (2008), http://www.q-impress.eu/

  6. Becker, S., Dencker, T., Happe, J.: Model-driven generation of performance prototypes. In: Kounev, S., Gorton, I., Sachs, K. (eds.) SIPEW 2008. LNCS, vol. 5119, pp. 79–98. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  7. Becker, S., Koziolek, H., Reussner, R.: The Palladio Component Model for Model-driven Performance Prediction. J. Syst. Softw. 82(1) (2009)

    Google Scholar 

  8. Bertolino, A.: Software Testing Research: Achievements, Challenges, Dreams. In: Proc. Intl. Conf. on Software Engineering, ICSE 2007, W. on the Future of Software Engineering, FOSE 2007. IEEE CS, Los Alamitos (2007)

    Google Scholar 

  9. Budinsky, F., Brodsky, S.A., Merks, E.: Eclipse Modeling Framework. Pearson Education, London (2003)

    Google Scholar 

  10. Cascaval, C., DeRose, L., Padua, D.A., Reed, D.A.: Compile-Time Based Performance Prediction. In: Carter, L., Ferrante, J. (eds.) LCPC 1999. LNCS, vol. 1863, p. 365. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  11. Franks, G., Maly, P., Woodside, M., Petriu, D.C., Hubbard, A.: Layered Queueing Network Solver and Simulator User Manual (2005), http://www.sce.carleton.ca/rads/lqns/

  12. Franks, G., Al-Omari, T., Woodside, M., Das, O., Derisavi, S.: Enhanced Modeling and Solution of Layered Queueing Networks. IEEE Trans. Software Eng. 35(2) (2009)

    Google Scholar 

  13. Grundy, J.C., Cai, Y., Liu, A.: Generation of Distributed System Test-Beds from High-Level Software Architecture Descriptions. In: Proc. 16th IEEE Intl. Conf. on Automated Software Engineering, ASE 2001. IEEE CS, Los Alamitos (2001)

    Google Scholar 

  14. Henning, J.L.: SPEC CPU2006 Benchmark Descriptions. SIGARCH Comput. Archit. News 34(4) (2006)

    Google Scholar 

  15. Hrischuk, C.E., Rolia, J.A., Woodside, C.M.: Automatic Generation of a Software Performance Model Using an Object-Oriented Prototype. In: Proc. 3rd Intl. W. on Modeling, Analysis, and Simulation On Computer and Telecommunication Systems, MASCOTS 1995. IEEE CS, Los Alamitos (1995)

    Google Scholar 

  16. Joshi, A., Eeckhout, L., Bell Jr., R.H., John, L.K.: Distilling the Essence of Proprietary Workloads Into Miniature Benchmarks. ACM Trans. Archit. Code Optim. 5(2) (2008)

    Google Scholar 

  17. Kounev, S.: Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri Nets. IEEE Trans. Software Eng. 32(7) (2006)

    Google Scholar 

  18. Kounev, S., Buchmann, A.: SimQPN: A Tool and Methodology for Analyzing Queueing Petri Net Models by Means of Simulation. Perform. Eval. 63(4) (2006)

    Google Scholar 

  19. Koziolek, H., Happe, J., Becker, S.: Parameter dependent performance specifications of software components. In: Hofmeister, C., Crnković, I., Reussner, R. (eds.) QoSA 2006. LNCS, vol. 4214, pp. 163–179. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  20. Liu, Y., Fekete, A., Gorton, I.: Design-Level Performance Prediction of Component-Based Applications. IEEE Trans. Software Eng. 31(11) (2005)

    Google Scholar 

  21. Liu, Y., Gorton, I.: Accuracy of Performance Prediction for EJB Applications: A Statistical Analysis. In: Gschwind, T., Mascolo, C. (eds.) SEM 2004. LNCS, vol. 3437, pp. 185–198. Springer, Heidelberg (2005)

    Google Scholar 

  22. OW2 Consortium: RUBiS: Rice University Bidding System, http://rubis.ow2.org/

  23. Rausch, A., Reussner, R., Mirandola, R., Plasil, F. (eds.): The Common Component Modeling Example: Comparing Software Component Models. Springer, Heidelberg (2008)

    Google Scholar 

  24. Standard Performance Evaluation Corporation: SPEC CPU2006 Benchmark, http://www.spec.org/cpu2006/

  25. Standard Performance Evaluation Corporation: SPECjAppServer2004 Benchmark, http://www.spec.org/jAppServer2004/

  26. Sun Microsystems, Inc.: Java Pet Store Demo, http://blueprints.dev.java.net/petstore/index.html

  27. The Q-ImPrESS Project Consortium: Quality Impact Prediction for Evolving Service-oriented Software, http://www.q-impress.eu/

  28. Transaction Processing Performance Council: TPC Benchmarks, http://www.tpc.org/information/benchmarks.asp

  29. Weyuker, E.J., Vokolos, F.I.: Experience with Performance Testing of Software Systems: Issues, an Approach, and Case Study. IEEE Trans. Software Eng. 26(12) (2000)

    Google Scholar 

  30. Woodside, C.M., Neron, E., Ho, E.D.S., Mondoux, B.: An “Active Server” Model for the Performance of Parallel Programs Written Using Rendezvous. J. Syst. Softw. 6(1-2) (1986)

    Google Scholar 

  31. Woodside, C.M., Schramm, C.: Scalability and Performance Experiments Using Synthetic Distributed Server Systems. Distributed Systems Engineering 3(1) (1996)

    Google Scholar 

  32. Xu, J., Oufimtsev, A., Woodside, M., Murphy, L.: Performance Modeling and Prediction of Enterprise JavaBeans with Layered Queuing Network Templates. SIGSOFT Softw. Eng. Notes 31(2) (2006)

    Google Scholar 

  33. Zhu, L., Gorton, I., Liu, Y., Bui, N.B.: Model Driven Benchmark Generation for Web Services. In: Proc. 2006 Intl. W. on Service-oriented Software Engineering, SOSE 2006. ACM, New York (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Distributed and Dependable Systems, Faculty of Mathematics and Physics, Charles University, Malostranské náměstí 25, 118 00, Prague, Czech Republic

    Vlastimil Babka, Petr Tůma & Lubomír Bulej

  2. Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod Vodárenskou věží 2, 182 07, Prague, Czech Republic

    Lubomír Bulej

Authors
  1. Vlastimil Babka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petr Tůma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lubomír Bulej
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, Worcester Polytechnic Institute, 100 Institute Road, 01609, Worcester, MA, USA

    George T. Heineman

  2. Faculty of Mathematics and Physiscs, Charles University in Prague, Malostranske namesti 25, 118 00, Praha 1, Czech Republic

    Jan Kofron

  3. Faculty of Mathematics and Physics, Charles University in Prague, Malostranske namesti 25, 11800, Prague 1, Czech Republic

    Frantisek Plasil

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Babka, V., Tůma, P., Bulej, L. (2010). Validating Model-Driven Performance Predictions on Random Software Systems. In: Heineman, G.T., Kofron, J., Plasil, F. (eds) Research into Practice – Reality and Gaps. QoSA 2010. Lecture Notes in Computer Science, vol 6093. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13821-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-13821-8_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-13820-1

  • Online ISBN: 978-3-642-13821-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation