Software & Systems Modeling

, Volume 14, Issue 4, pp 1505–1526 | Cite as

A framework for qualitative assessment of domain-specific languages

Regular Paper


Domain-specific languages (DSLs) are used for improving many facets of software development, but whether and to what extent this aim is achieved is an important issue that must be addressed. This paper presents a proposal for a Framework for Qualitative Assessment of DSLs (FQAD). FQAD is used for determining the perspective of the evaluator, understanding the goal of the assessment and selecting fundamental DSL quality characteristics to guide the evaluator in the process. This framework adapts and integrates the ISO/IEC 25010:2011 standard, CMMI maturity level evaluation approach and the scaling approach used in DESMET into a perspective-based assessment. A detailed list of domain-specific language quality characteristics is elaborated, and a novel assessment method is proposed. Two case studies through which FQAD is matured and evaluated are reported. The case studies have shown that stakeholders find the FQAD process beneficial.


Domain-specific languages Quality measures Qualitative assessment ISO/IEC 25010 CMMI 


  1. 1.
    Amstel, M.F., Lange, C.F., Brand, M.G.: Using Metrics for Assessing the Quality of ASF+SDF Model Transformations, ICMT. Springer, Berlin (2009)Google Scholar
  2. 2.
    Bunge, M.A., Ontology, I.: The Furniture of the World (vol. 3). Reidel, Dordrecht (1977)Google Scholar
  3. 3.
    Cameron, K.S.: Critical questions in assessing organizational effectiveness. Organ. Dyn. 9(2), 66–80 (1980)CrossRefGoogle Scholar
  4. 4.
    Chen, Y., Dios, R., Mili, A., Wu, L., An Empirical Study of Programming Language Trends. New Jersey Institute of Technology, Kefei Wang, State University of New York, Albany, IEEE (2005)Google Scholar
  5. 5.
    Frank, U.: Domain-specific modeling languages-requirements analysis and design guidelines. In: Reinhartz-Berger, I., Sturm, A., Clark, T., Wand, Y., Cohen, S., Bettin, J. (eds.) Domain Engineering: Product Lines, Conceptual Models, and Languages. Springer, Berlin (2013)Google Scholar
  6. 6.
    Gabriel, P.H.N.: Software Langauges Engineering: Experimental Evaluation. Dissertacao apresentada na Faculdade Ciencias e Tecnologia da Universidade Nova de Lisboa para obtencao do grau de Mestre em Engenharia Informatica, Lisboa (2010)Google Scholar
  7. 7.
    Haugen, O., Mohagheghi, P.: A multi-dimensional framework for characterizing domain specific languages. In: Proceeding of the 7th OOPSLA Workshop on Domain Specific Modeling (2007)Google Scholar
  8. 8.
    Hermans, F., Pinzger, M., Deursen, A.V.: Domain specific langages in practice: a user study on the success factors. In: MODELS09, pp. 423–437. Springer, Berlin (2009)Google Scholar
  9. 9.
    ISO/IEC 25010:2011: Systems and Software Engineering Systems and Software Quality Requirements and Evaluation (SQuaRE) System and Software Quality Models, International Standards Organization/International Electrotechnical Commission (2011)Google Scholar
  10. 10.
    Kahlaoui, A., Abran, A., Lefebvre, E.: DSML success factors and their assessment criteria. Metrics News 13(1), 43–51 (2008)Google Scholar
  11. 11.
    Kärna, J., Tolvanen, J.P., Kelly, S.: Evaluating the use of domain-specific modeling in practice. In: Proceedings of DSM09 (2009)Google Scholar
  12. 12.
    Karsai, G., Krahn, H., Pinkernell, C., Rumpe, B., Schindler, M., Volkel, S.: Design guidelines for domain specific languages. In: Proceedings of the 9th OOPSLA Workshop on Domain-Specific Modeling (DSM’ 09), Orlando, FL, USA (October 2009)Google Scholar
  13. 13.
    Kelly, S., Tolvanen, J.P.: Domain Specific Modeling Enabling Full Code Generation. Wiley, New York (2008)Google Scholar
  14. 14.
    Kelly, S., Pohjonen, R.: Worst practices for domain-specific modeling. IEEE Softw. 26(4), 22–29, (July/August 2009)Google Scholar
  15. 15.
    Khedker, U.P.: What Makes a Good Programming Language? Technical Report TR-97-upk-1, Department of Computer Science University of Pune (1997)Google Scholar
  16. 16.
    Kitchenham, B.A., Linkman, S., Law, D.: DESMET: a methodology for evaluating software engineering methods and tools. Comput. Control Eng. J. 8(3), 120–126 (1997)Google Scholar
  17. 17.
    Kolovos, D.S., Paige, R.F., Kelly, T.P., Polack, F.A.C.: Requirements for domain-specific languages. In: Proceedings of the First ECOOP Workshop on Domain-Specific Program Development, Co-located with ECOOP06, Nantes, France (2006)Google Scholar
  18. 18.
    Kosar, T., Oliviera, N., Mernik, M., Pereira, V.M.J., Crepinsek, M., Cruz, D., Henriques, P.R.: Comparing general purpose and domain specific languages: an empirical study. ComSIS 7(2), 247–264 (Special Issue) (2010)Google Scholar
  19. 19.
    Kouhen, A.E., Dumoulin, C., Gerard, S., Boulet, P.: Evaluation of Modeling Tools Adaptation, hal-00706701, version 2 (2012) Google Scholar
  20. 20.
    Krogstie, J.: Evaluating UML Using a Generic Quality Framework Chapter in UML and the Unified Process. Idea Group Publishing, Hershey (2003)Google Scholar
  21. 21.
    Merilinna, J., Pärssinen, J.: Comparison between different abstraction level programming: experiment definition and initial results. In: OOPSLA Workshop on Domain-Specific Modeling, Montreal, Canada (2007)Google Scholar
  22. 22.
    McKean, D., Sprinkle, J.: Heterogeneous multi-core systems: UML profiles vs. DSM Approaches. In: Proceedings of the 2012 Workshop on Domain Specific Modeling (2012)Google Scholar
  23. 23.
    Mohagheghi, P., Dehlen, V., Neple, T.: Definitions and approaches to model quality in model based software development a review of literature. Inf. Softw. Technol. 51, 1646–1669 (2009)CrossRefGoogle Scholar
  24. 24.
    Moody, D.L.: The physics of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35, 756–779 (2009)CrossRefGoogle Scholar
  25. 25.
    Paige, R., Ostroff, J., Brooke, P.: Principles for Modeling Language Design, Techical Report CS-1999-08, York University (1999)Google Scholar
  26. 26.
    Pfleeger, S.L., Kitchenham, B.A.: Principles of survey research. ACM SIGSOFT Softw. Eng. Notes 26, 16–18 (2001)CrossRefGoogle Scholar
  27. 27.
    Runeson, P., Host, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14–2, 131–164 (2009)CrossRefGoogle Scholar
  28. 28.
    Software Engineering Institute, CMMI for Acquisition, Version 1.3, Technical Report, CMU/SEI-2010-TR-032, Carnegie Mellon (November 2012)Google Scholar
  29. 29.
    Strembeck, M., Zdun, U.: An approach for the systematic development of domain specific languages. Softw. Pract. Exp. 39, 1253–1292 (2009); John Wiley & Sons LtdGoogle Scholar
  30. 30.
    Wand, Y., Weber, R.: On the ontological expressiveness of information systems analysis and design grammars. J. Inf. Syst. 3, 217–237 (1993)CrossRefGoogle Scholar
  31. 31.
    Wand, Y., Weber, R.: On the deep structure of information systems. Inf. Syst. J. 5, 203–223 (1995)CrossRefGoogle Scholar
  32. 32.
    Wu, Y., Hernandez, F., Ortega, F., Clarke, P.J., France, R.: Measuring the effort for creating and using domain specific models. In: Proceedings of 10th Domain Specific Modeling Workshop (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Department of Electrical and Electronics EngineeringMiddle East Technical UniversityAnkaraTurkey

Personalised recommendations