Advertisement

Empirical Software Engineering

, Volume 23, Issue 3, pp 1743–1790 | Cite as

On the impact of state-based model-driven development on maintainability: a family of experiments using UniMod

  • Filippo Ricca
  • Marco Torchiano
  • Maurizio Leotta
  • Alessandro Tiso
  • Giovanna Guerrini
  • Gianna Reggio
Article

Abstract

Context: Model-driven approaches are well-known in the academia but one possible showstopper to a wider adoption in the industry is the limited empirical evidence for their claimed advantages and benefits, that could convince the decision makers. Objective: The aim of this work is gauging one of the claimed benefits of model-driven approaches, namely improvement in maintainability, with respect to a code-centric approach. Method: We conducted a family of five experiments involving 100 students that possessed different levels of education (64 Bachelor, 25 Master, and 11 PhD students; in groups sized 11 to 26 per individual experiment). In these experiments, UniMod – a State-based tool for Model-Driven Development using the Unified Modeling Language – is compared with Java-based code-centric programming, in a software maintenance scenario, with the goal of analyzing the effect on the time to perform the maintenance tasks, the correctness of the modified artifacts, and the efficiency. Results: The results show a reduction in time to accomplish the tasks and no impact on correctness. The adoption of the UniMod-MDD approach almost doubles the developers’efficiency, and in presence of a higher software engineering experience the efficiency is even three times higher. Conclusions: We found that the usage of the UniMod-MDD approach in a software maintenance scenario provides benefits over a pure code-centric approach. The benefits deriving from the UniMod-MDD approach are appreciable for all the categories of students, although with differences.

Keywords

Model-driven development MDD UML Maintainability UniMod Family of experiments Replication and maintenance task 

References

  1. Abdi H (2007) Bonferroni and Sidak corrections for multiple comparisons. Sage, Thousand OaksGoogle Scholar
  2. Agner LTW, Soares IW, Stadzisz PC, Simão JM (2013) A Brazilian survey on UML and model-driven practices for embedded software development. J Syst Softw 86(4):997–1005CrossRefGoogle Scholar
  3. Arisholm E, Briand L, Hove S, Labiche Y (2006) The impact of UML documentation on software maintenance: an experimental evaluation. IEEE Trans Softw Eng 32(6):365–381CrossRefGoogle Scholar
  4. Anda B, Hansen K, Gullesen I, Thorsen HK (2006) Experiences from introducing UML-based development in a large safety-critical project. Empir Softw Eng 11(4):555–581.  https://doi.org/10.1007/s10664-006-9020-6 CrossRefGoogle Scholar
  5. Baker R (1995) Modern permutation test software. In: Edgington E (ed) Randomization Tests, Marcel DeckerGoogle Scholar
  6. Ceccato M, Penta M, Falcarin P, Ricca F, Torchiano M, Tonella P (2014) A family of experiments to assess the effectiveness and efficiency of source code obfuscation techniques. Empirx Softw Eng 19(4):1040–1074. [Online]. Available:  https://doi.org/10.1007/s10664-013-9248-x Google Scholar
  7. Ceri S, Fraternali P, Bongio A, Brambilla M, Comai S, Matera M (2002) Designing data-intensive web applications. Morgan Kaufmann Publishers Inc., San FranciscoGoogle Scholar
  8. Chapin N, Hale JE, Kham KM, Ramil JF, Tan W-G (2001) Types of software evolution and software maintenance. J Softw Maint 13(1):3–30. [Online]. Available: http://dl.acm.org/citation.cfm?id=371697.371701 CrossRefzbMATHGoogle Scholar
  9. Dzidek WJ, Arisholm E, Briand LC (2008) A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans Softw Eng 34 (3):407–432.  https://doi.org/10.1109/TSE.2008.15 CrossRefGoogle Scholar
  10. Feitelson DG (2015) Using students as experimental subjects in software engineering research - A review and discussion of the evidence, CoRR, vol. abs/1512.08409. [Online]. Available: 1512.08409
  11. Fowler M (2003) UML distilled: a brief guide to the standard object modeling language, 3rd edn. Addison-Wesley Longman Publishing Co., Inc., BostonGoogle Scholar
  12. Grissom RJ, Kim JJ (2005) Effect sizes for research: A broad practical approach, 2nd edn. Lawrence Earlbaum Associates, New JerseyGoogle Scholar
  13. Gurov V, Mazin M, Narvsky A, Shalyto A (2007) Tools for support of automata-based programming. Programm Comput Softw 33:343–355CrossRefzbMATHGoogle Scholar
  14. Hamon G (2005) A denotational semantics for stateflow. In: Proceedings of the 5th ACM International Conference on Embedded Software, ser. EMSOFT ’05. ACM, New York, pp 164–172. [Online]. Available:  https://doi.org/10.1145/1086228.1086260
  15. Hamon G, Rushby J (2007) An operational semantics for stateflow. Int J Softw Tools Technol Transf 9(5):447–456. [Online]. Available:  https://doi.org/10.1007/s10009-007-0049-7 CrossRefzbMATHGoogle Scholar
  16. Harel D (1987) Statecharts: a visual formalism for complex systems. S Comput Programm 8(3):231–274. [Online]. Available: http://www.sciencedirect.com/science/article/pii/0167642387900359 MathSciNetCrossRefzbMATHGoogle Scholar
  17. Heiberger R, Robbins N (2014) Design of diverging stacked bar charts for likert scales and other applications. J Stat Softw 57(5):1–32, 4. [Online]. Available: http://www.jstatsoft.org/v57/i05 CrossRefGoogle Scholar
  18. Hutchinson J., Whittle J, Rouncefield M, Kristoffersen S (2011) Empirical assessment of MDE in industry. In: Proceedings of 33rd International Conference on Software Engineering (ICSE 2011). ACM, New York, pp 471–480Google Scholar
  19. Hovsepyan A, Scandariato R, Van Baelen S, Joosen W, Demeyer S (2011) Preserving aspects via automation: A maintainability study. In: Proceedings of 5th International Symposium on Empirical Software Engineering and Measurement (ESEM 2011). IEEE CS, Washington, pp 315–324. [Online]. Available:  https://doi.org/10.1109/ESEM.2011.40
  20. Jedlitschka A, Ciolkowski M, Pfahl D (2008) Reporting experiments in software engineering. In: Guide to Advanced Empirical Software Engineering. Springer, BerlinGoogle Scholar
  21. Kapteijns T, Jansen S, Brinkkemper S, Houet H, Barendse R (2009) A comparative case study of model driven development vs traditional development: the tortoise or the hare. In: 4th European Workshop on “From code centric to model centric software engineering: Practices, Implications and ROI” (C2M 2009). CTIT Workshop Proceedings Series, pp 22–33Google Scholar
  22. Kitchenham B, Al-Khilidar H, Babar M, Berry M, Cox K, Keung J, Kurniawati F, Staples M, Zhang H, Zhu L (2008) Evaluating guidelines for reporting empirical software engineering studies. Empir Softw Eng 13:97–121CrossRefGoogle Scholar
  23. Kleppe AG, Warmer J, Bast W (2003) MDA explained: the model driven architecture: practice and promise. Addison-Wesley Longman Publishing Co. Inc, BostonGoogle Scholar
  24. Leotta M, Ricca F, Antoniol G, Garousi V, Zhi J, Ruhe G (2013) A pilot experiment to quantify the effect of documentation accuracy on maintenance tasks. In: Proceedings of 29th International Conference on Software Maintenance (ICSM 2013). IEEE, pp 428–431.  https://doi.org/10.1109/ICSM.2013.64
  25. Liebel G, Marko N, Tichy M, Leitner A, Hansson J (2014) Assessing the state-of-practice of model-based engineering in the embedded systems domain. In: Dingel J, Schulte W, Ramos I, Abrahão S, Insfran E (eds) Proceedings of 17th International Conference on Model Driven Engineering Languages and Systems (MODELS 2014), ser. Lecture Notes in Computer Science, vol 8767. Springer, Berlin, pp 166–182. [Online]. Available:  https://doi.org/10.1007/978-3-319-11653-2_11
  26. Martínez Y, Cachero C, Matera M, Abrahao S, Luján S (2011) Impact of MDE approaches on the maintainability of web applications: an experimental evaluation. In: Proceedings of 30th International Conference on Conceptual Modeling (ER 2011). Springer-Verlag, Berlin, pp 233–246. [Online]. Available: http://dl.acm.org/citation.cfm?id=2075144.2075168
  27. Martínez Y, Cachero C, Meliá S (2014) Empirical study on the maintainability of web applications: Model-driven engineering vs code-centric. Empir Softw Eng 19 (6):1887–1920. [Online]. Available:  https://doi.org/10.1007/s10664-013-9269-5 CrossRefGoogle Scholar
  28. Maraninchi F, Rémond Y (2001) Argos: an automaton-based synchronous language. Comput Lang 27(1–3):61–92. visual Formal Methods-VFM’99 Symposium. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0096055101000169 CrossRefzbMATHGoogle Scholar
  29. Mellegård N, Ferwerda A, Lind K, Heldal R, Chaudron MRV (2016) Impact of introducing domain-specific modelling in software maintenance: An industrial case study. IEEE Trans Softw Eng 42(3):245–260CrossRefGoogle Scholar
  30. Melia S, Gomez J, Perez S, Diaz O (2008) A model-driven development for gwt-based rich internet applications with ooh4ria. In: Proceedings of 8th International Conference on Web Engineering (ICWE 2008), pp 13–23Google Scholar
  31. Motulsky H (2010) Intuitive biostatistics: a non-mathematical guide to statistical thinking. Oxford University Press, Oxford. [Online]. Available: http://books.google.it/books?id=R477U5bAZs4C Google Scholar
  32. Mussbacher G, Amyot D, Breu R, Bruel J-M, Cheng B, Collet P, Combemale B, France R, Heldal R, Hill J, Kienzle J, Schöttle M, Steimann F, Stikkolorum D, Whittle J (2014) The relevance of model-driven engineering thirty years from now. In: Dingel J., Schulte W., Ramos I., Abrahão S, Insfran E (eds) Proceedings of 17th International Conference on Model Driven Engineering Languages and Systems (MODELS 2014), ser. Lecture Notes in Computer Science, vol 8767. Springer, Berlin, pp 183–200. [Online]. Available:  https://doi.org/10.1007/978-3-319-11653-2_12
  33. Oppenheim AN (1992) Questionnaire Design, Interviewing and Attitude Measurement. Pinter, LondonGoogle Scholar
  34. Papotti P, do Prado A, de Souza W, Cirilo C, Pires L (2013) A quantitative analysis of model-driven code generation through software experimentation. In: Salinesi C, Norrie M, Pastor Ó (eds) Advanced Information Systems Engineering, ser. Lecture Notes in Computer Science, vol 7908. Springer, Berlin Heidelberg, pp 321–337. [Online]. Available:  https://doi.org/10.1007/978-3-642-38709-8_21
  35. Reggio G, Leotta M, Ricca F (2014) Who knows/uses what of the UML: A personal opinion survey. In: Dingel J., Schulte W, Ramos I., Abrahão S, Insfran E (eds) Proceedings of 17th International Conference on Model Driven Engineering Languages and Systems (MODELS 2014), ser. Lecture Notes in Computer Science, vol 8767. Springer, Berlin, pp 149–165.  https://doi.org/10.1007/978-3-319-11653-2_10
  36. Reggio G, Leotta M, Ricca F, Clerissi D (2015) What are the used UML diagram constructs? A document and tool analysis study covering activity and use case diagrams. In: Hammoudi S, Pires LF, Filipe J, das Neves RC (eds) Model-Driven Engineering and Software Development, ser. Communications in Computer and Information Science, vol 506. Springer, Berlin, pp 66–83. [Online]. Available:  https://doi.org/10.1007/978-3-319-25156-1_5
  37. Pfleeger SL, Menezes W (2000) Marketing technology to software practitioners. IEEE Softw 17(1):27–33CrossRefGoogle Scholar
  38. Ricca F, Di Penta M, Torchiano M, Tonella P, Ceccato M, Visaggio CA (2008) Are Fit tables really talking?: a series of experiments to understand whether fit tables are useful during evolution tasks. In: Proceedings of 30th International Conference on Software Engineering (ICSE 2008). ACM, New York, pp 361–370. [Online]. Available:  https://doi.org/10.1145/1368088.1368138
  39. Ricca F, Di Penta M, Torchiano M, Tonella P, Ceccato M (2010) How developers’ experience and ability influence web application comprehension tasks supported by UML stereotypes: A series of four experiments. IEEE Trans Softw Eng 36:96–118CrossRefGoogle Scholar
  40. Ricca F, Leotta M, Reggio G, Tiso A, Guerrini G, Torchiano M (2012) Using UniMod for maintenance tasks: An experimental assessment in the context of model driven development. In: Proceedings of 4th International Workshop on Modeling in Software Engineering (MiSE 2012). IEEE, pp 77–83. [Online]. Available:  https://doi.org/10.1109/MISE.2012.6226018
  41. R Core Team R (2013) A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. ISBN 3-900051-07-0. [Online]. Available: http://www.R-project.org/ Google Scholar
  42. Sheskin D (2007) Handbook of parametric and nonparametric statistical procedures, 4th edn. Chapman & All, UKzbMATHGoogle Scholar
  43. Shalyto AA, Tukkel NI (2001) SWITCH technology: an automated approach to developing software for reactive systems. Programm Comput Softw 27:260–276CrossRefzbMATHGoogle Scholar
  44. Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52:3MathSciNetCrossRefzbMATHGoogle Scholar
  45. Sommerville I (2006) Software engineering. Addison-Wesley Longman Publishing Co. Inc, BostonzbMATHGoogle Scholar
  46. Stahl T, Voelter M, Czarnecki K (2006) Model-driven software development: technology, engineering, management. Wiley, New JerseyGoogle Scholar
  47. Svahnberg M, Aurum A, Wohlin C (2008) Using students as subjects-an empirical evaluation. In: Proceedings of 2nd International Symposium on Empirical Software Engineering and Measurement (ESEM 2008), pp 288–290Google Scholar
  48. Torchiano M, Tomassetti FCA, Ricca F, Tiso A, Reggio G (2013) Relevance, benefits, and problems of software modelling and model driven techniques–a survey in the Italian industry. J Syst Softw 86(8):2110–2126CrossRefGoogle Scholar
  49. Torchiano M (2015) effsize: Efficient effect size computation. r package version 0.5.5. [Online]. Available: http://CRAN.R-project.org/package=effsize
  50. Torchiano M, Scanniello G, Ricca F, Reggio G, Leotta M (2017) Do UML object diagrams affect design comprehensibility? Results from a family of four controlled experiments. J Vis Lang Comput 41:10–21. [Online]. Available:  https://doi.org/10.1016/j.jvlc.2017.06.002 CrossRefGoogle Scholar
  51. Wheeler B (2016) lmPerm: Permutation tests for linear models. R package version 2.0. [Online]. Available: http://CRAN.R-project.org/package=lmPerm
  52. Whittle J, Hutchinson J, Rouncefield M (2014) The state of practice in model-driven engineering. IEEE Softw 31(3):79–85CrossRefGoogle Scholar
  53. Wohlin C, Runeson P, Höst M, Ohlsson M, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer-Verlag, Berlin HeidelbergCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.DIBRISUniversità di GenovaGenovaItaly
  2. 2.DAUINPolitecnico di TorinoTorinoItaly

Personalised recommendations