Advertisement

MBT4J: Automating the Model-Based Testing Process for Java Applications

  • Leonardo Villalobos-Arias
  • Christian Quesada-López
  • Alexandra Martinez
  • Marcelo Jenkins
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 865)

Abstract

Model-based testing is a process that can reduce the cost of software testing by automating the design and generation of test cases but it usually involves some time-consuming manual steps. Current model-based testing tools automate the generation of test cases, but offer limited support for the model creation and test execution stages. In this paper we present MBT4J, a platform that automates most of the model-based testing process for Java applications, by integrating several existing tools and techniques. It automates the model building, test case generation, and test execution stages of the process. First, a model is extracted from the source code, then an adapter—between this model and the software under test—is generated and finally, test cases are generated and executed. We performed an evaluation of our platform with 12 configurations using an existing Java application from a public repository. Empirical results show that MBT4J is able to generate up to 2,438 test cases, detect up to 289 defects, and achieve a code coverage ranging between 72% and 84%. In the future, we plan to expand our evaluation to include more software applications and perform error seeding in order to be able to analyze the false positive and negative rates of our platform. Improving the automation of oracles is another vein for future research.

Keywords

MBT4J platform Model-based testing Software testing Model extraction Test generation Test execution Adapter Automation 

References

  1. 1.
    Utting, M., Legeard, B., Bouquet, F., Fourneret, E., Peureux, F., Vernotte, A.: Recent advances in model-based testing. In: Advances in Computers, vol. 101, pp. 53–120. Elsevier (2016)Google Scholar
  2. 2.
    Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann, San Francisco (2010)Google Scholar
  3. 3.
    Pretschner, A., Prenninger, W., Wagner, S., Kühnel, C., Baumgartner, M., Sostawa, B., Zölch, R., Stauner, T.: One evaluation of model-based testing and its automation. In: Proceedings of the 27th International Conference on Software Engineering, pp. 392–401. ACM (2005)Google Scholar
  4. 4.
    Broy, M., Jonsson, B., Katoen, J.P., Leucker, M., Pretschner, A.: Model-based testing of reactive systems. In: LNCS, vol. 3472. Springer (2005)Google Scholar
  5. 5.
    Jard, C., Jeron, T.: TGV: theory, principles and algorithms. Int. J. Softw. Tools Technol. Transfer 7, 297–315 (2005)CrossRefGoogle Scholar
  6. 6.
    SwissQ: Testing trends & benchmarks 2013. where do we stand where are we going to? (2013) https://swissq.it/wp-content/uploads/2016/02/Testing-Trends_und_Benchmarks2013.pdf
  7. 7.
    Schulze, C., Ganesan, D., Lindvall, M., Cleaveland, R., Goldman, D.: Assessing model-based testing: an empirical study conducted in industry. In: Proceedings of the 36th International Conference on Software Engineering. ACM (2014)Google Scholar
  8. 8.
    Ernits, J., Roo, R., Jacky, J., Veanes, M.: Model-based testing of web applications using NModel. In: Testing of Software Communication Systems. Springer (2009)Google Scholar
  9. 9.
    Pinheiro, A.C., Simao, A., Ambrosio, A.M.: FSM-based test case generation methods applied to test the communication software on board the ITASAT university satellite: a case study. J. Aerosp. Technol. Manag. 6, 447–461 (2014)Google Scholar
  10. 10.
    de Cleva Farto, G., Endo, A.T.: Evaluating the model-based testing approach in the context of mobile applications. Electron. Notes Theor. Comput. Sci. 314, 3–21 (2015)CrossRefGoogle Scholar
  11. 11.
    Villalobos-Arias, L., Quesada-López, C., Martinez, A., Jenkins, M.: A tertiary study on model-based testing areas, tools and challenges: preliminary results. In: 21st Ibero-American Conference on Software Engineering (2018)Google Scholar
  12. 12.
    Hashemi Aghdam, A.: Generating test adapters for ModelJunit (2017)Google Scholar
  13. 13.
    Mariani, L., Pezze, M., Zuddas, D.: Augusto: exploiting popular functionalities for the generation of semantic GUI tests with oracles (2018)Google Scholar
  14. 14.
    Legeard, B., Peureux, F., Utting, M.: Automated boundary testing from z and b. In: International Symposium of Formal Methods Europe, pp. 21–40. Springer (2002)Google Scholar
  15. 15.
    Hessel, A., Pettersson, P.: Cover-a real-time test case generation tool. In: 19th IFIP International Conference on Testing of Communicating Systems and 7th International Workshop on Formal Approaches to Testing of Software (2007)Google Scholar
  16. 16.
    Berghofer, S., Nipkow, T.: Random testing in Isabelle/HOL. In: SEFM (2004)Google Scholar
  17. 17.
    Belinfante, A.: JTorX: a tool for on-line model-driven test derivation and execution. In: International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 266–270. Springer (2010)Google Scholar
  18. 18.
    Veanes, M., Campbell, C., Grieskamp, W., Schulte, W., Tillmann, N., Nachmanson, L.: Model-based testing of object-oriented reactive systems with spec explorer. In: Formal Methods and Testing, pp. 39–76 (2008)Google Scholar
  19. 19.
    Barnett, M., Grieskamp, W., Nachmanson, L., Schulte, W., Tillmann, N., Veanes, M.: Towards a tool environment for model-based testing with ASML. In: International Workshop on Formal Approaches to Software Testing. Springer (2003)Google Scholar
  20. 20.
    Jacky, J.: Pymodel: model-based testing in python. In: Proceedings of the Python for Scientific Computing Conference (2011)Google Scholar
  21. 21.
    Blom, J., Jonsson, B., Nyström, S.O.: Industrial evaluation of test suite generation strategies for model-based testing. In: 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation (ICSTW), pp. 209–218. IEEE (2016)Google Scholar
  22. 22.
    Amalfitano, D., Amatucci, N., Memon, A.M., Tramontana, P., Fasolino, A.R.: A general framework for comparing automatic testing techniques of android mobile apps. J. Syst. Softw. 125, 322–343 (2017)CrossRefGoogle Scholar
  23. 23.
    Duarte, L.M., Kramer, J., Uchitel, S.: Using contexts to extract models from code. Softw. Syst. Model. 16, 523–557 (2017)CrossRefGoogle Scholar
  24. 24.
    Duarte, L.M.: Behaviour model extraction using context information. PhD thesis, Department of Computing, Imperial College London (2007)Google Scholar
  25. 25.
    Cordy, J., Carmichael, I., Promislow, E.: The TXL programming language (2017). https://www.txl.ca/. Accessed 15 June 2018
  26. 26.
    Magee, J., Krammer, J., Chatley, R., Uchitel, S., Foster, H.: LTSA—labelled transition system analyser (2006). https://www.doc.ic.ac.uk/ltsa/. Accessed 19 June 2018
  27. 27.
    Blmaster: Java source: Calculator (2008). http://forum.codecall.net/topic/42522-java-source-code-calculator-app/. Accessed 15 June 2018

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Leonardo Villalobos-Arias
    • 1
  • Christian Quesada-López
    • 1
  • Alexandra Martinez
    • 1
  • Marcelo Jenkins
    • 1
  1. 1.University of Costa RicaSan PedroCosta Rica

Personalised recommendations