Quality in model-driven engineering: a tertiary study
- First Online:
Model-driven engineering (MDE) is believed to have a significant impact in software quality. However, researchers and practitioners may have a hard time locating consolidated evidence on this impact, as the available information is scattered in several different publications. Our goal is to aggregate consolidated findings on quality in MDE, facilitating the work of researchers and practitioners in learning about the coverage and main findings of existing work as well as identifying relatively unexplored niches of research that need further attention. We performed a tertiary study on quality in MDE, in order to gain a better understanding of its most prominent findings and existing challenges, as reported in the literature. We identified 22 systematic literature reviews and mapping studies and the most relevant quality attributes addressed by each of those studies, in the context of MDE. Maintainability is clearly the most often studied and reported quality attribute impacted by MDE. Eighty out of 83 research questions in the selected secondary studies have a structure that is more often associated with mapping existing research than with answering more concrete research questions (e.g., comparing two alternative MDE approaches with respect to their impact on a specific quality attribute). We briefly outline the main contributions of each of the selected literature reviews. In the collected studies, we observed a broad coverage of software product quality, although frequently accompanied by notes on how much more empirical research is needed to further validate existing claims. Relatively, little attention seems to be devoted to the impact of MDE on the quality in use of products developed using MDE.
KeywordsQuality Model-driven engineering Tertiary study
- Ameller, D., Franch, X., Gómez, C., Araujo, J., Svensson, R. B., Biffl, S., Cabot, J., Cortellessa, V., Daneva, M., Fernández, D. M., Moreira, A., Muccini, H., Vallecillo, A., Wimmer, M., Amaral, V., Brunelièrek, H., Burgueño, L., Goulão, M., Schätz, B., & Teufl, S. (2015) Handling non-functional requirements in model-driven development: An ongoing industrial survey. In 23rd IEEE international requirements engineering conference (RE 2015).Google Scholar
- Badreddin, O., Lethbridge, T. C., & Elassar, M. (2013). Modeling practices in open source software. In Open source software: Quality verification (pp. 127–139). Springer. doi:10.1007/978-3-642-38928-3_9.
- Badreddin, O., Sturm, A., Hamou-Lhadj, A., Lethbridge, T., Dixon, W., & Simmons, R. (2015). The effects of education on students’ perception of modeling in software engineering. In First International workshop on human factors in modeling (HuFaMo 2015), CEUR workshop proceedingsGoogle Scholar
- Biolchini, J., Mian, P. G., Natali, A. C. C., & Travassos, G. H. (2005). Systematic review in software engineering. Tech. Rep. RT–ES 679/05, System engineering and computer science department COPPE/UFRJ. http://www.cin.ufpe.br/~in1037/leitura/systematicReviewSE-COPPE.pdf.
- Cuadrado, J. S., Molina, J. G., & Tortosa, M. M. (2006). Rubytl: A practical, extensible transformation language. In A. Rensink & J. Warmer (Eds.), Model driven architecture–Foundations and applications (pp. 158–172). Bilbao, Spain: Springer.Google Scholar
- Dybå, T., & Dingsøyr, T. (2008). Strength of evidence in systematic reviews in software engineering. In Proceedings of the second acm-ieee international symposium on empirical software engineering and measurement (pp. 178–187), ACM. doi:10.1145/1414004.1414034.
- Fernández-Sáez, A. M., Genero, M., Caivano, D., & Chaudron, M. R. V. (2015). On the use of UML documentation in software maintenance: Results from a survey in industry. In ACM/IEEE 18th International conference on model driven engineering languages and systems (MODELS 2015) (pp. 292–301). ACM/IEEE.Google Scholar
- Forward, A., Lethbridge, T., & Badreddin, O. (2010). Problems and opportunities for model-centric vs code-centric development: A survey of software professionals. In 5th Workshop “From code centric to model centric: Evaluating the effectiveness of MDD (C2M: EEMDD)”, University Pierre & Marie Curie, Paris.Google Scholar
- Hutchinson, J., Whittle, J., Rouncefield, M., & Kristoffersen, S. (2011). Empirical assessment of MDE in industry. In Proceedings of the 33rd international conference on software engineering (pp. 471–480). ACM, New York, NY, USA, ICSE ’11. doi:10.1145/1985793.1985858.
- ISO/IEC. (2011). IEC25010:2011 Systems and software engineering–Systems and software Quality Requirements and Evaluation (SQuaRE)–System and software quality models. International Organization for Standardization, 34. Google Scholar
- Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. In Technical report, Version 2.3 EBSE technical report. EBSE, Keele University and Durham University Joint Report. http://www.dur.ac.uk/ebse/resources/Systematic-reviews-5-8.pdf.
- Kitchenham, B. A., Dyba, T., & Jorgensen, M. (2004). Evidence-based software engineering. In Proceedings of the 26th international conference on software engineering (pp. 273–281). IEEE Computer Society. doi:10.1109/ICSE.2004.1317449.
- OMG (2015). OMG unified modeling language ™(OMG UML). Tech. Rep. formal/2015-03-01, Object Management Group. http://www.omg.org/spec/UML/2.5.
- Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008). Systematic mapping studies in software engineering. In 12th international conference on evaluation and assessment in software engineering Google Scholar
- Petre, M. (2013). Uml in practice. In D. Notkin, B. H. C. Cheng, & K. Pohl (Eds.), Proceedings of the 2013 international conference on software engineering (pp. 722–731). San Francisco, CA: IEEE Press. doi:10.1109/ICSE.2013.6606618.
- Petticrew, M., & Roberts, H. (2008). Systematic reviews in the social sciences: A practical guide. New York: Wiley.Google Scholar
- Whittle, J., Hutchinson, J., Rouncefield, M., Burden, H., & Heldal, R. (2015). A taxonomy of tool-related issues affecting the adoption of model-driven engineering. Software and Systems Modeling, pp. 1–19. doi:10.1007/s10270-015-0487-8.
- Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering (pp. 38:1–38:10). ACM, New York, NY, USA, EASE ’14. doi:10.1145/2601248.2601268.
- Zhou, Y., Zhang, H., Huang, X., Yang, S., Babar, M. A., Tang, H. (2015). Quality assessment of systematic reviews in software engineering: A tertiary study. In Proceedings of the 19th international conference on evaluation and assessment in software engineering (pp. 14:1–14:14). ACM, New York, NY, USA, EASE ’15. doi:10.1145/2745802.2745815.
Included Secondary Studies
- Delgado, A., Ruiz, F., de Guzmán, I. G. R., & Piattini, M. (2013). Main principles on the integration of SOC and MDD paradigms to business processes: A systematic review. In J. Cordeiro, M. Virvou, & B. Shishkov (Eds.), Software and Data Technologies (pp. 88–108). Athens, Greece: Springer. doi:10.1007/978-3-642-29578-2_6.
- Giachetti, G., Valverde, F., & Marín, B. (2012). Interoperability for model-driven development: Current state and future challenges. In Sixth international conference on research challenges in information science (RCIS 2012) (pp. 1–10). IEEE. doi:10.1109/RCIS.2012.6240445.
- Giraldo, F. D., España, S., & Pastor, O. (2014). Analysing the concept of quality in model-driven engineering literature: A systematic review. In Eighth international conference on research challenges in information science (RCIS 2014), (pp. 1–12). IEEE. doi:10.1109/RCIS.2014.6861030.
- Giraldo, F. D., España, S., Giraldo, W. J., & Pastor, O. (2015). Modelling language quality evaluation in model-driven information systems engineering: a roadmap. In 9th International conference on research challenges in information science (RCIS 2015) (pp. 64–69). IEEE. doi:10.1109/RCIS.2015.7128864.
- Hansson, S., Zhao, Y., & Burden, H. (2014). How MAD are we? empirical evidence for model-driven agile development. In 3rd Workshop on Extreme Modeling, XM 2014, CEUR Workshop Proceedings (pp. 2–11). vol 1239. URL http://ceur-ws.org/Vol-1239/XM2014#page=9.
- Jensen, J., & Jaatun, M. G. (2011) Security in model driven development: A survey. In Availability, reliability and security (ARES), 2011 Sixth International conference on (pp. 704–709). IEEE. doi:10.1109/ARES.2011.110.
- Loniewski, G., Insfran, E., & Abrahão, S. (2010). A systematic review of the use of requirements engineering techniques in model-driven development. In Model driven engineering languages and systems (pp. 213–227). Springer. doi:10.1007/978-3-642-16129-216.
- Malavolta, I., & Muccini, H. (2014). A study on MDE approaches for engineering wireless sensor networks. In Software engineering and advanced applications (SEAA), 2014 40th EUROMICRO conference on (pp. 149–157). IEEE. doi:10.1109/SEAA.2014.61.
- Mohagheghi, P., & Dehlen, V. (2008). Where is the proof?—a review of experiences from applying mde in industry. In Schieferdecker, I., Hartman, A. (eds) Model driven architecture—foundations and applications, Lecture notes in computer Science (pp. 432–443). vol 5095, Springer, Berlin. doi:10.1007/978-3-540-69100-631.
- Neto, V. V. G., Guessi, M., Oliveira, L. B. R., Oquendo, F., & Nakagawa, E. Y. (2014). Investigating the model-driven development for systems-of-systems. In Proceedings of the 2014 European Conference on software architecture workshops (pp. 22:1–22:8). ACM, New York, NY, USA, ECSAW ’14. doi:10.1145/2642803.2642825.
- Nguyen, P. H., Klein, J., le Traon, Y., & Kramer, M. E. (2013) A systematic review of model-driven security. In 20th Asia–Pacific software engineering conference (APSEC 2013) (pp. 432–441). vol 1. doi:10.1109/APSEC.2013.64.
- Santiago, I., Jiménez, Á., Vara, J. M., De Castro, V., Bollati, V. A., & Marcos, E. (2012). Model-driven engineering as a new landscape for traceability management: A systematic literature review. Information and Software Technology, 54(12), 1340–1356. doi:10.1016/j.infsof.2012.07.008.CrossRefGoogle Scholar
- Szvetits, M., & Zdun, U. (2013). Systematic literature review of the objectives, techniques, kinds, and architectures of models at runtime. Software and Systems Modeling, pp. 1–39. doi:10.1007/s10270-013-0394-9.