Abstract
Predicting the performance of software architectures during early design stages is an active field of research in software engineering. It is expected that accurate predictions minimize the risk of performance problems in software systems by a great extent. This would improve quality and save development time and costs of subsequent code fixings. Although a lot of different methods have been proposed, none of them have gained widespread application in practice. In this paper we describe the evaluation and comparison of three approaches for early performance predictions (Software Performance Engineering (SPE), Capacity Planning (CP) and umlPSI). We conducted an experiment with 31 computer science students. Our results show that SPE and CP are suited for supporting performance design decisions in our scenario. CP is also able to validate performance goals as stated in requirement documents under certain conditions. We found that SPE and CP are matured, yet lack the proper tool support that would ease their application in practice.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Anderberg, M.R.: Cluster Analysis for Applications. Academic Press, London (1973)
Balsamo, S., DiMarco, A., Inverardi, P., Simeoni, M.: Model-based performance prediction in software development: A survey. IEEE Transactions on Software Engineering 30(5), 295–310 (2004)
Balsamo, S., Marzolla, M., DiMarco, A., Inverardi, P.: Experimenting different software architectures performance techniques: A case study. In: Proceedings of the Fourth International Workshop on Software and Performance, pp. 115–119. ACM Press, New York (2004)
Basili, V.R., Caldiera, G., Rombach, H.D.: The goal question metric approach. Encyclopedia of Software Engineering 2, 528–532 (1994)
Bertolino, A., Mirandola, R.: CB-SPE tool: Putting component-based performance engineering into practice. In: Crnkovic, I., Stafford, J.A., Schmidt, H.W., Wallnau, K.C. (eds.) CBSE 2004. LNCS, vol. 3054, pp. 233–248. Springer, Heidelberg (2004)
Gorton, I., Liu, A.: Performance evaluation of alternative component architectures for enterprise javabean applications. IEEE Internet Computing 7(3), 18–23 (2003)
Koziolek, H.: Empirische bewertung von performance-analyseverfahren für software-architekturen. Diploma thesis, University of Oldenburg, Faculty II, Department of Computing Science, Okt (2004)
Marzolla, M.: Simulation-Based Performance Modeling of UML Software Architectures. PhD thesis, Universit‘a Ca Foscari di Venezia (2004)
Menascé, D.A., Almeida, V.A.F., Dowdy, L.W.: Download files for the book: Performance by design: computer capacity planning by example (2004), http://cs.gmu.edu/~menasce/perfbyd/efiles.html
Menasce, D.A., Almeida, V.A.F., Dowdy, L.W.: Performance by Design. Prentice-Hall, Englewood Cliffs (2004)
Menasce, D.A., Almeida, V.A.F.: Capacity Planning for Web Services. Prentice-Hall, Englewood Cliffs (2002)
Object Management Group OMG. Uml profile for schedulability, performance and time (2003), http://www.omg.org/cgi-bin/doc?formal/2003-09-01
Smith, C.U.: Speed: The software performance engineering (spe) tool (January 2000), http://www.perfeng.com/sped.htm
Smith, C.U.: Performance Solutions: A Practical Guide To Creating Responsive, Scalable Software. Addison-Wesley, Reading (2002)
Smith, C.U.: Spe-ed user guide (2003), http://www.perfeng.com/papers/manual.zip
Smith, C.U., Lladó, C.M.: Performance model interchange format (pmif 2.0): Xml definition and implementation. Technical report, Performance Engineering Services, Universitat Illes Balears (2004)
Wohling, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslen, A.: Experimentation in Software Engineering – An Introduction. Kluwer Academic Publishers, Dordrecht (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Koziolek, H., Firus, V. (2005). Empirical Evaluation of Model-Based Performance Prediction Methods in Software Development. In: Reussner, R., Mayer, J., Stafford, J.A., Overhage, S., Becker, S., Schroeder, P.J. (eds) Quality of Software Architectures and Software Quality. QoSA SOQUA 2005 2005. Lecture Notes in Computer Science, vol 3712. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11558569_14
Download citation
DOI: https://doi.org/10.1007/11558569_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29033-9
Online ISBN: 978-3-540-32056-2
eBook Packages: Computer ScienceComputer Science (R0)