A systematic approach to evaluating domain-specific modeling language environments for multi-agent systems
Multi-agent systems (MASs) include multiple interacting agents within an environment to provide a solution for complex systems that cannot be easily solved with individual agents or monolithic systems. However, the development of MASs is not trivial due to the various agent properties such as autonomy, responsiveness, and proactiveness, and the need for realization of the many different agent interactions. To support the development of MASs various domain-specific modeling languages (DSMLs) have been introduced that provide a declarative approach for modeling and supporting the generation of agent-based systems. To be effective, the proposed DSMLs need to meet the various stakeholder concerns and the related quality criteria for the corresponding MASs. Unfortunately, very often the evaluation of the DSML is completely missing or has been carried out in idiosyncratic approach. If the DSMLs are not well defined, then implicitly this will have an impact on the quality of the MASs. In this paper, we present an evaluation framework and systematic approach for assessing existing or newly defined DSMLs for MASs. The evaluation is specific for MAS DSMLs and targets both the language and the corresponding tools. To illustrate the evaluation approach, we first present SEA_ML, which is a model-driven MAS DSML for supporting the modeling and generation of agent-based systems. The evaluation of SEA_ML is based on a multi-case study research approach and provides both qualitative evaluation and quantitative analysis. We report on the lessons learned considering the adoption of the evaluation approach as well as the SEA_ML for supporting the generation of agent-based systems.
KeywordsDomain-specific modeling language (DSML) Evaluation framework Evaluation of DSMLs Multi-agent system (MAS) Software agent SEA_ML
- AOS. (2001). Agent Oriented Software Pty., Ltd. JACK Environment. http://www.aosgrp.com/products/jack/. Last Accessed August 2015.
- Barisic, A. (2013). Evaluating the quality in use of domain-specific languages in an agile way. In Proceedings of the doctoral symposium at 16th international conference on model driven engineering languages and systems (MODELS), September, 2013, CEUR-WS.Google Scholar
- Barisic, A., Amaral, V., Goulao, M., & Barroca, B. (2011a). How to reach a usable DSL? Moving toward a systematic evaluation. Electronic Communications of the EASST, 50, 1–12.Google Scholar
- Barisic, A., Amaral, V., Goulao, M., & Barroca, B. (2011b). Quality in use of domain-specific languages: A case study. In Proceedings of the 3rd ACM SIGPLAN workshop on evaluation and usability of programming languages and tools (pp. 65–72). PLATEAU’11, New York, USA.Google Scholar
- Barisic, A., Amaral, V., Goulao, M., & Barroca, B. (2012a). Evaluating the usability of domain-specific languages. In M. Mernik (Ed.), Formal and practical aspects of domain-specific languages: Recent developments (pp. 386–407). Hershey: IGI Global.Google Scholar
- Barisic, A., Amaral, V., Goulao, M., & Monteiro, M. P. (2012b). Patterns for evaluating usability of domain-specific languages. In Proceedings of the 19th conference on pattern languages of programs (PLoP), SPLASH.Google Scholar
- Bobkowska, A. (2005). A methodology of visual modeling language evaluation. In SOFSEM 2005: Theory and practice of computer systems. Lecture Notes in Computer Science (Vol. 3381).Google Scholar
- Clark, T., Evans, A., Sammut, P., & Willans, J. (2004). Language driven development and MDA. MDA Journal, 2–13. http://www.bptrends.com/bpt/wp-content/publicationfiles/10-04%20COL%20MDA%20-%20Frankel%20-%20Xactium.pdf. Last Accessed August 2015.
- CMU/SEI. (2010). Software Engineering Institute, CMMI for Acquisition, Version 1.3, Technical Report, CMU/SEI-2010-TR-032, Carnegie Mellon (November 2012).Google Scholar
- Demirkol, S., Challenger, M., Getir, S., Kosar, T., Kardas, G., & Mernik, M. (2013). A DSL for the development of software agents working within a semantic web environment. Computer Science and Information Systems, Special Issue on Advances in Model Driven Engineering, Languages and Agents, 10(4), 1525–1556.Google Scholar
- Demirkol, S., Getir, S., Challenger, M., & Kardas, G. (2011). Development of an agent based E-barter System. In: International symposium on innovations in intelligent systems and applications (INISTA) (pp. 193–198). IEEE Computer Society, Istanbul-Turkey.Google Scholar
- Demirli, E., & Tekinerdogan, B. (2011). Software language engineering of architectural viewpoints. In Proceeding of the 5th European Conference on Software Architecture (ECSA 2011), Lecture Notes on Computer Science (Vol. 6903, pp. 336–343).Google Scholar
- Eclipse. (2004). The Eclipse Foundation, Eclipse Integrated Development Environment (IDE). http://www.eclipse.org/. Last Accessed August 2015.
- Fowler, M. (2011). Domain-specific languages. Boston: Addison-Wesley.Google Scholar
- Garcia, A. F., Sant’Anna, C., Figueiredo, E., Kulesza, U., Pereira de Lucena, C. J., & von Staa, A. (2005). Modularizing design patterns with aspects: A quantitative study. In Proceedings of the 4th international conference on Aspect-oriented software development (AOSD 2005) (pp. 3–14).Google Scholar
- Getir, S., Challenger, M., Demirkol, S., & Kardas, G. (2012). The semantics of the interaction between agents and web services on the semantic web. In Proceedings of the 7th IEEE international workshop on engineering semantic agent systems (ESAS 2012), held in conjunction with the 36th IEEE signature conference on computers, software, and applications (COMPSAC 2012) (pp. 619–624).Google Scholar
- Gray, J., Tolvanen, J.-P., Kelly, S., Gokhale, A., Neema, S., & Sprinkle, J. (2007). Domain-specific modeling. In P. A. Fishwick (Ed.), Handbook of dynamic system modeling (pp. 1–7). Boca Raton: CRC Press.Google Scholar
- Hahn, C. (2008). A domain specific language for multiagent systems. In Proceedings of the 7th autonomous agents and multiagent systems conference (AAMAS’08), Estoril, Portugal (pp. 233–240).Google Scholar
- Hahn, C., Nesbigall, S., Warwas, S., Zinnikus, I., Fischer, K., & Klusch, M. (2008). Integration of multiagent systems and semantic web services on a platform independent level. In Proceedings of IEEE/WIC/ACM international conference on web intelligence and intelligent agent technology (WI-IAT 2008), Sydney, Australia (pp. 200–206).Google Scholar
- Hannemann, J., & Kiczales, G. (2002). Design patterns implementation in Java and AspectJ. In Proceeding of object oriented programming systems languages and applications (OOPSLA’02) (pp. 161–173).Google Scholar
- ISO/IEC. (1998). International Organization for Standardization (Ed.), ISO/IEC TR 15504-2. Information Technology-Software Process Assessment—Part 2: A reference model for processes and process capability. Case Postale 56, CH-1211 Geneva, Switzerland.Google Scholar
- ISO/IEC. (2011). ISO/IEC 25010. Systems and software engineering systems and software quality requirements and evaluation (SQuaRE), System and software quality models, International Standards Organization/International Electrotechnical Commission.Google Scholar
- Kahlaoui, A., Abran, A., & Lefebvre, E. (2008). DSML success factors and their assessment criteria. Metrics News, 13(1), 43–51.Google Scholar
- Khedker, U. P. (1997). What makes a good programming language? Technical Report TR-97-upk-1, Department of Computer Science, University of Pune.Google Scholar
- Kirstan, S., & Zimmermann, J. (2010). Evaluating costs and benefits of model-based development of embedded software systems in the car industry—Results of a qualitative Case Study. In Proceedings of ECMFA 2010 workshop C2 M:EEMDD-from Code Centric to Model Centric: Evaluating the effectiveness of MDD (pp 18–29).Google Scholar
- Martin, D., Burstein, M., Hobbs, J., Lassila, O., McDermott, D., McIlraith, S., Narayanan, S., Paolucci, M., Parsia, B., Payne, T., Sirin, E., Srinivasan, N., & Sycara, K. (2004). OWL-S: Semantic markup for web services. W3C Member Submission. http://www.w3.org/Submission/OWL-S/. Last Accessed August 2015.
- MOFScript. (2005). The Eclipse Foundation, MOFScript model-to-text transformation language and tool. http://www.eclipse.org/gmt/mofscript/. Last Accessed August 2015.
- Mohagheghi, P., & Dehlen, V. (2008). Where is the proof? A review of experiences from applying MDE in industry. In: I. Schieferdecker, & A. Hartman (Eds.) ECMDA-FA 2008. Lecture notes in computer science (Vol. 5095, pp. 432–443).Google Scholar
- Monteiro, M. P., & Fernandes, J. M. (2005). Towards a catalog of aspect oriented refactorings. In Proceedings of the 4th international conference on aspect oriented software development (AOSD’05) (pp. 111–122).Google Scholar
- OCL. (2012). Object Management Group, Object Constraint Language (OCL). http://www.omg.org/spec/OCL/2.3.1/. Last Accessed August 2015.
- Rao, A., & Georgeff, M. (1995). BDI agents: From theory to practice. In Proceedings of the 1st international conference on multi-agent systems (ICMAS-95), San Francisco (pp. 312–319).Google Scholar
- Rougemaille, S., Migeon, F., Maurel, C., & Gleizes, M.-P. (2007). Model driven engineering for designing adaptive multi-agent systems. Lecture Notes in Artificial Intelligence, 4995, 318–332.Google Scholar
- Schmid, K., & John, I. (2002). Developing, validating and evolving an approach to product line benefit and risk assessment. In Proceedings of the 28th Euro-micro conference (EUROMICRO’02) (pp. 272–283).Google Scholar
- Sycara, K. (1998). Multiagent systems. AI Magazine, 19(4), 79–92.Google Scholar
- Tekinerdogan, B., Bozbey, S., Mester, Y., Turançiftci, E., & Alkışlar, L. (2011). An aspect-oriented tool framework for developing process-sensitive embedded user assistance systems. In Transactions on aspect-oriented software development VIII. Lecture notes computer science (Vol. 6580, pp. 196–220).Google Scholar
- Tekinerdogan, B., & Demirli, E. (2013). Evaluation framework for software architecture viewpoint languages. In Proceedings of 9th international ACM Sigsoft conference on the quality of software architectures (QoSA 2013) (pp. 89–98).Google Scholar
- Warmer, J., & Kleppe, A. G. (1998). The object constraint language: Precise modeling with UML. Boston: Addison-Wesley.Google Scholar