Model-Based Regression Testing of Autonomous Robots

  • Dávid Honfi
  • Gábor Molnár
  • Zoltán MicskeiEmail author
  • István Majzik
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10567)


Testing is a common technique to assess quality of systems. Regression testing comes into view, when changes are introduced to the system under test and re-running all tests is not practical. Numerous techniques have been introduced to select tests only relevant to a given set of changes. These are typically based on source code, however, model-based development projects use models as primary artifacts described in various domain-specific languages. Thus, regression test selection should be performed directly on these models. We present a method and a case study on how model-based regression testing can be achieved in the context of autonomous robots. The method uses information from several domain-specific languages for modeling the robot’s context and configuration. Our approach is implemented in a prototype tool, and its scalability is evaluated on models from the case study.



This work was partially supported by the ARTEMIS JU and the Hungarian National Research, Development and Innovation Fund in the frame of the R5-COP project.


  1. 1.
    Aggrawal, K., Singh, Y., Kaur, A.: Code coverage based technique for prioritizing test cases for regression testing. ACM Softw. Eng. Notes 29(5), 1–4 (2004)CrossRefGoogle Scholar
  2. 2.
    Agrawal, H., Horgan, J.R., Krauser, E.W., London, S.: Incremental regression testing. Int. Conf. Softw. Maintenance 93, 348–357 (1993)Google Scholar
  3. 3.
    Almasri, N., Tahat, L., Korel, B.: Toward automatically quantifying the impact of a change in systems. Softw. Qual. J., 1–40 (2016)Google Scholar
  4. 4.
    Altmanninger, K., Seidl, M., Wimmer, M.: A survey on model versioning approaches. Int. J. Web Inform. Syst. 5(3), 271–304 (2009)CrossRefGoogle Scholar
  5. 5.
    ASTM International: Standard Terminology for Evaluating Response Robot Capabilities E2521–16 (2016)Google Scholar
  6. 6.
    Bergmann, G., Dávid, I., Hegedüs, Á., Horváth, Á., Ráth, I., Ujhelyi, Z., Varró, D.: Viatra 3: a reactive model transformation platform. In: Kolovos, D., Wimmer, M. (eds.) ICMT 2015. LNCS, vol. 9152, pp. 101–110. Springer, Cham (2015). doi: 10.1007/978-3-319-21155-8_8 CrossRefGoogle Scholar
  7. 7.
    Brambilla, M., Cabot, J., Wimmer, M.: Model-Driven Software Engineering in Practice, 1st edn. Morgan & Claypool Publishers, Williston (2012)Google Scholar
  8. 8.
    Briand, L., Labiche, Y., He, S.: Automating regression test selection based on UML designs. Inf. Softw. Technol. 51(1), 16–30 (2009)CrossRefGoogle Scholar
  9. 9.
    Briand, L., Labiche, Y., Soccar, G.: Automating impact analysis and regression test selection based on UML designs. In: International Conference on Software Maintenance, pp. 252–261 (2002)Google Scholar
  10. 10.
    Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Conference of the Centre for Advanced Studies on Collaborative Research, pp. 1–14 (2002)Google Scholar
  11. 11.
    Chen, Y., Probert, R.L., Ural, H.: Regression test suite reduction using extended dependence analysis. In: 4th International Workshop on Software Quality Assurance, SOQUA 2007, pp. 62–69. ACM (2007)Google Scholar
  12. 12.
    Connelly, J., Hong, W., Mahoney, R., Sparrow, D.: Challenges in autonomous system development. In: Proceedings of Performance Metrics for Intelligent Systems Workshop (PerMIS 2006) (2006)Google Scholar
  13. 13.
    Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Inf. Softw. Technol. 52(1), 14–30 (2010)CrossRefGoogle Scholar
  14. 14.
    Farooq, Q., Iqbal, M., Malik, Z., Riebisch, M.: A model-based regression testing approach for evolving software systems with flexible tool support. In: IEEE International Conference on Engineering of Computer Based Systems, pp. 41–49 (2010)Google Scholar
  15. 15.
    Farooq, Q.u.a., Iqbal, M.Z.Z., Malik, Z.I., Nadeem, A.: An approach for selective state machine based regression testing. In: Proceeding of the 3rd International Workshop on Advances in Model-based Testing, A-MOST, pp. 44–52. ACM (2007)Google Scholar
  16. 16.
    Fourneret, E., Cantenot, J., Bouquet, F., Legeard, B., Botella, J.: SeTGaM: generalized technique for regression testing based on UML/OCL models. In: International Conference on Software Security and Reliability, pp. 147–156. IEEE, US (2014)Google Scholar
  17. 17.
    Graves, T.L., Harrold, M.J., Kim, J.M., Porter, A., Rothermel, G.: An empirical study of regression test selection techniques. ACM TOSEM 10(2), 184–208 (2001)CrossRefzbMATHGoogle Scholar
  18. 18.
    Guiochet, J., Machin, M., Waeselynck, H.: Safety-critical advanced robots: a survey. Robot. Auton. Syst. 94, 43–52 (2017)CrossRefGoogle Scholar
  19. 19.
    Harman, M.: Making the case for MORTO: multi objective regression test optimization. In: ICST Workshops, pp. 111–114 (2011)Google Scholar
  20. 20.
    Harrold, M.J., Gupta, R., Soffa, M.L.: A methodology for controlling the size of a test suite. ACM TOSEM 2(3), 270–285 (1993)CrossRefGoogle Scholar
  21. 21.
    Harrold, M.J., Jones, J.A., Li, T., Liang, D., Orso, A., Pennings, M., Sinha, S., Spoon, S.A., Gujarathi, A.: Regression test selection for Java software. ACM SIGPLAN Not. 36(11), 312–326 (2001)CrossRefGoogle Scholar
  22. 22.
    IEEE: Systems and software engineering - Vocabulary, standard 24765:2010 (2010)Google Scholar
  23. 23.
    Jacoff, A., Huang, H.M., Messina, E., Virts, A., Downs, A.: Comprehensive standard test suites for the performance evaluation of mobile robots. In: Proc of the 10th Performance Metrics for Intelligent Systems Workshop, PerMIS 2010, pp. 161–168. ACM (2010)Google Scholar
  24. 24.
    Korel, B., Tahat, L., Vaysburg, B.: Model based regression test reduction using dependence analysis. In: International Conference on Software Maintenance, pp. 214–223 (2002)Google Scholar
  25. 25.
    Le Traon, Y., Jeron, T., Jezequel, J., Morel, P.: Efficient object-oriented integration and regression testing. IEEE Tran. Reliab. 49(1), 12–25 (2000)CrossRefGoogle Scholar
  26. 26.
    Leung, H., White, L.: Insights into regression testing. In: International Conference on Software Maintenance, pp. 60–69, October 1989Google Scholar
  27. 27.
    Malishevsky, A.G., Ruthruff, J.R., Rothermel, G., Elbaum, S.: Cost-cognizant test case prioritization. Technical report, Department of Computer Science and Engineering, University of Nebraska-Lincoln (2006)Google Scholar
  28. 28.
    Micskei, Z., Szatmári, Z., Oláh, J., Majzik, I.: A concept for testing robustness and safety of the context-aware behaviour of autonomous systems. In: Jezic, G., Kusek, M., Nguyen, N.-T., Howlett, R.J., Jain, L.C. (eds.) KES-AMSTA 2012. LNCS, vol. 7327, pp. 504–513. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-30947-2_55 CrossRefGoogle Scholar
  29. 29.
    NIST: Guide for Evaluating, Purchasing, and Training with Response Robots using DHS-NIST-ASTM International Standard Test Methods (2014).
  30. 30.
    Orso, A., Do, H., Rothermel, G., Harrold, M.J., Rosenblum, D.S.: Using component metadata to regression test component-based software. Softw. Testing Verification Reliab. 17(2), 61–94 (2007)CrossRefGoogle Scholar
  31. 31.
    Pilskalns, O., Uyan, G., Andrews, A.: Regression testing UML designs. In: International Conference on Software Maintenance, pp. 254–264 (2006)Google Scholar
  32. 32.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing (2013).
  33. 33.
    R5-COP: Incremental testing of behaviour (2016)., d34.20 deliverable
  34. 34.
    R5-COP: Assessment of the On-line Verification and Incremental Testing (2017)., d34.50 deliverable
  35. 35.
    Rothermel, G., Harrold, M.J.: Selecting regression tests for object-oriented software. In: International Conference on Software Maintenance, pp. 14–25. IEEE (1994)Google Scholar
  36. 36.
    Rothermel, G., Harrold, M.J.: Analyzing regression test selection techniques. IEEE Tran. Softw. Eng. 22(8), 529–551 (1996)CrossRefGoogle Scholar
  37. 37.
    Rothermel, G., Untch, R.H., Chu, C., Harrold, M.J.: Prioritizing test cases for regression testing. IEEE Tran. Softw. Eng. 27(10), 929–948 (2001)CrossRefGoogle Scholar
  38. 38.
    Soetens, Q.D., Demeyer, S.: ChEOPSJ: change-based test optimization. In: European Conference on Software Maintenance and Reengineering, pp. 535–538 (2012)Google Scholar
  39. 39.
    de Sousa Santos, I., de Castro Andrade, R.M., Rocha, L.S., Matalonga, S., de Oliveira, K.M., Travassos, G.H.: Test case design for context-aware applications: are we there yet? Inf. Softw. Technol. 88, 1–16 (2017)CrossRefGoogle Scholar
  40. 40.
    Tengeri, D., Beszedes, A., Havas, D., Gyimothy, T.: Toolset and program repository for code coverage-based test suite analysis and manipulation. In: 14th IEEE International Working Conference on Source Code Analysis and Manipulation, pp. 47–52 (2014)Google Scholar
  41. 41.
    Vaysburg, B., Tahat, L.H., Korel, B.: Dependence analysis in reduction of requirement based test suites. In: Proceeding of the International Symposium on Software Testing and Analysis, pp. 107–111 (2002)Google Scholar
  42. 42.
    Wu, Y., Offutt, J.: Maintaining evolving component-based software with UML. In: European Conference on Software Maintenance and Reengineering, pp. 133–142 (2003)Google Scholar
  43. 43.
    Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Testing Verification Reliab. 22(2), 67–120 (2012)CrossRefGoogle Scholar
  44. 44.
    Zech, P., Felderer, M., Kalb, P., Breu, R.: A generic platform for model-based regression testing. In: Margaria, T., Steffen, B. (eds.) ISoLA 2012. LNCS, vol. 7609, pp. 112–126. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-34026-0_9 CrossRefGoogle Scholar
  45. 45.
    Zech, P., Kalb, P., Felderer, M., Atkinson, C., Breu, R.: Model-based regression testing by OCL. Int. J. STTT 19, 115–131 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Dávid Honfi
    • 1
  • Gábor Molnár
    • 1
  • Zoltán Micskei
    • 1
    Email author
  • István Majzik
    • 1
  1. 1.Department of Measurement and Information SystemsBudapest University of Technology and EconomicsBudapestHungary

Personalised recommendations