Abstract
Model driven software engineering (MDSE) has shown to provide mark improvement in productivity and quality of software products. UML is a standard modeling language that is widely used in the industry to support MDSE. To provide tool support for MDSE, a large number of UML modeling tools are available, ranging from open-source tools to commercial tools with high price tag. A common decision faced while applying UML in practice is the selection of an appropriate tool for modeling. In this paper we conduct a study to compare three of the well-known modeling tools: IBM Rational Software Architect (RSA), MagicDraw, and Papyrus. In this study we conducted an experiment with undergraduate and graduate students. The goal is to compare the productivity of the software engineers while modeling with the tools. We measure the productivity in terms of modeling effort required to correctly complete a task, learnability, time and number of clicks required, and memory load required for the software engineer to complete a task. Our results show that MagicDraw performed significantly better in terms of learnability, memory load, and completeness of tasks. In terms of time and number of clicks, IBM RSA was significantly better while modeling class diagrams and state machines when compared to Papyrus. However no single tool outperformed others in all the modeling tasks with respect to time and number of clicks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Iqbal, M.Z., Ali, S., Yue, T., Briand, L.: Applying UML/MARTE on industrial projects: challenges, experiences, and guidelines. Software & Systems Modeling, 1–19 (2014)
Khaled, L.: A comparison between UML tools. In: Second International Conference on Environmental and Computer Science, pp. 111–114. IEEE (2009)
Eichelberger, H., Eldogan, Y., Schmid, K.: A Comprehensive Survey of UML Compliance in Current Modelling Tools. Software Engineering 143, 39–50 (2009)
Heena, R.: A comparative study of UML tools. In: Proceedings of 11th International Conference on Advances in Computing and Artificial Intelligence, pp. 1–4 (2011)
Bellon, S., Koschke, R., Antoniol, G., Krinke, J., Merlo, E.: Comparison and evaluation of clone detection tools. IEEE Transactions on Software Engineering 33, 577–591 (2007)
Ali, S., Yue, T., Briand, L.: Assessing quality and effort of applying aspect state machines for robustness testing: a controlled experiment. In: IEEE 6th International Conference on Software Testing, Verification and Validation, pp. 212–221. IEEE (2013)
Dias Neto, A.C., Subramanyan, R., Vieira, M., Travassos, G.H.: A survey on model-based testing approaches: a systematic review. In: Proceedings of the 1st ACM International Workshop on Empirical Assessment of Software Engineering Languages and Technologies: Held in Conjunction with ASE Conference, pp. 31–36. ACM (2007)
Sengoz, Y.S., Jawawi, A., Deris, S.B.: A Comparison UML Tools for Real-time Software Modeling (2009)
Rani, T., Garg, S.: Comparison of different UML tool: Tool approach. International Journal Of Engineering And Computer Science 2, 1900–1908 (2013)
Bobkowska, A.E., Reszke, K.: Usability of UML modeling tools. In: Proceedings of the Conference on Software Engineering: Evolution and Emerging Technologies, pp. 75–86 (2005)
Smith, H.H.: On tool selection for illustrating the use of UML in system development. Journal of Computing Sciences in Colleges 19, 53–63 (2004)
Ali, S., Iqbal, M.Z., Arcuri, A., Briand, L.: A search-based OCL constraint solver for model-based test data generation. In: 11th International Conference on Quality Software (QSIC), pp. 41–50. IEEE (2011)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in software engineering. Springer (2012)
Usman, M., Iqbal, M.Z., Khan, M.U.: A model-driven approach to generate mobile applications for multiple platforms. In: Asia-Pacific Software Engineering Conference (2014)
Jilani, A.A., Iqbal, M.Z., Khan, M.U.: A search based test data generation approach for model transformations. In: Di Ruscio, D., Varró, D. (eds.) ICMT 2014. LNCS, vol. 8568, pp. 17–24. Springer, Heidelberg (2014)
Khan, M.U., Iqbal, M.Z., Ali, S.: A Heuristic-based approach to refactor crosscutting behaviors in UML state machines. In: International Conference on Software Maintenance and Evolution. IEEE (2014)
Eriksson, H.-E., Penker, M., Lyons, B., Fado, D.: UML 2 toolkit. John Wiley & Sons (2003)
Binder, R.V.: Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Professional (2000)
Safdar, S.A., Iqbal, M.Z., Khan, M.U.: Empirical Evaluation of Productivity of Software Engineer in UML Modeling Tools- A Controlled Experiment. Technical report# 3515. Software Quality Engineering & Testing (QUEST) Lab (2015)
Sheskin, D.J.: Handbook of parametric and nonparametric statistical procedures. Chapman & Hall/CRC (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Safdar, S.A., Iqbal, M.Z., Khan, M.U. (2015). Empirical Evaluation of UML Modeling Tools–A Controlled Experiment. In: Taentzer, G., Bordeleau, F. (eds) Modelling Foundations and Applications. ECMFA 2015. Lecture Notes in Computer Science(), vol 9153. Springer, Cham. https://doi.org/10.1007/978-3-319-21151-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-21151-0_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21150-3
Online ISBN: 978-3-319-21151-0
eBook Packages: Computer ScienceComputer Science (R0)