Supporting inheritance hierarchy changes in model-based regression test selection

Theme Section Paper

Abstract

Models can be used to ease and manage the development, evolution, and runtime adaptation of a software system. When models are adapted, the resulting models must be rigorously tested. Apart from adding new test cases, it is also important to perform regression testing to ensure that the evolution or adaptation did not break existing functionality. Since regression testing is performed with limited resources and under time constraints, regression test selection (RTS) techniques are needed to reduce the cost of regression testing. Applying model-level RTS for model-based evolution and adaptation is more convenient than using code-level RTS because the test selection process happens at the same level of abstraction as that of evolution and adaptation. In earlier work, we proposed a model-based RTS approach called MaRTS to be used with a fine-grained model-based adaptation framework that targets applications implemented in Java. MaRTS uses UML models consisting of class and activity diagrams. It classifies test cases as obsolete, reusable, or retestable based on changes made to UML class and activity diagrams of the system being adapted. However, MaRTS did not take into account the changes made to the inheritance hierarchy in the class diagram and the impact of these changes on the selection of test cases. This paper extends MaRTS to support such changes and demonstrates that the extended approach performs as well as or better than code-based RTS approaches in safely selecting regression test cases. While MaRTS can generally be used during any model-driven development or model-based evolution activity, we have developed it in the context of runtime adaptation. We evaluated the extended MaRTS on a set of applications and compared the results with code-based RTS approaches that also support changes to the inheritance hierarchy. The results showed that the extended MaRTS selected all the test cases relevant to the inheritance hierarchy changes and that the fault detection ability of the selected test cases was never lower than that of the baseline test cases. The extended MaRTS achieved comparable results to a graph-walk code-based RTS approach (DejaVu) and showed a higher reduction in the number of selected test cases when compared with a static analysis code-based RTS approach (ChEOPSJ).

Keywords

Executable UML models Inheritance hierarchy Model-based adaptation Model-based regression test selection Model validation Runtime adaptation UML activity diagram UML class diagram 

Notes

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. CNS 1305381. We would like to acknowledge the support of Quinten Soetens who helped us install and run ChEOPSJ.

References

  1. 1.
    Al-Refai, M., Cazzola, W., Ghosh, S., France, R.: Using models to validate unanticipated, fine-grained adaptations at runtime. In: Waeselynck, H., Babiceanu, R.(eds.) Proceedings of the 17th IEEE International Symposium on High Assurance Systems Engineering (HASE’16), pp. 23–30. IEEE, Orlando, FL, USA (2016)Google Scholar
  2. 2.
    Al-Refai, M., Ghosh, S., Cazzola, W.: Model-based regression test selection for validating runtime adaptation of software systems. In: Briand, L., Khurshid, S. (eds.) Proceedings of the 9th IEEE international conference on software testing, verification and validation (ICST’16), pp. 288–298. IEEE, Chicago, IL, USA (2016)Google Scholar
  3. 3.
    Arcuri, A., Campos, J., Fraser, G.: Unit test generation during software development: EvoSuite Plugins for Maven, IntelliJ and Jenkins. In: Briand, L., Khurshid, S. (eds.) Proceedings of the 9th IEEE International Conference on Software Testing, Verification and Validation (ICST’16), pp. 401–408. IEEE, Chicago, IL, USA (2016)Google Scholar
  4. 4.
    Briand, L.C., Labiche, Y., He, S.: Automating regression test selection based on UML designs. J. Inf. Softw. Technol. 51(1), 16–30 (2009)CrossRefGoogle Scholar
  5. 5.
    Cazzola, W., Pini, S., Ghoneim, A., Saake, G.: Co-evolving application code and design models by exploiting meta-data. In: Proceedings of the 22nd Annual ACM Symposium on Applied Computing (SAC’07), pp. 1275–1279. ACM Press, Seoul, South Korea (2007)Google Scholar
  6. 6.
    Cazzola, W., Rossini, N.A., Al-Refai, M., France, R.B.: Fine-grained software evolution using UML activity and class models. In: Moreira, A., Schätz, B. (eds.) Proceedings of the 16th International Conference on Model Driven Engineering Languages and Systems (MoDELS’13), Lecture Notes in Computer Science 8107, pp. 271–286. Springer, Miami, FL, USA (2013)Google Scholar
  7. 7.
    Cazzola, W., Rossini, N.A., Bennett, P., Pradeep Mandalaparty, S., France, R.B.: Fine-grained semi-automated runtime evolution. In: Bencomo, N., Chang, B., France, R.B., Aßmann, U. (eds.) MoDELS@Run-Time, Lecture Notes in Computer Science 8378, pp. 237–258. Springer (2014)Google Scholar
  8. 8.
    Cazzola, W., Vacchi, E.: @Java: bringing a richer annotation model to Java. Comput. Lang. Syst. Struct. 40(1), 2–18 (2014). https://doi.org/10.1016/j.cl.2014.02.002 Google Scholar
  9. 9.
    Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Stewart, D.A., Johnson, J.H. (eds.) Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research (CASCON’02), pp. 1–14. IBM Press (2002)Google Scholar
  10. 10.
    Cottenier, T., van den Berg, A., Elrad, T.: Motorola WEAVR: aspect orientation and model-driven engineering. J. Obj. Technol. 6(7), 51–88 (2007)CrossRefGoogle Scholar
  11. 11.
    Dean, J., Grove, D., Chambers, C.: Optimization of object-oriented programs using static class hierarchy analysis. In: Olthoff, W.G. (ed.) Proceedings of the 9th European Conference on Object-Oriented Programming (ECOOP’95), LNCS 952, pp. 77–101. Springer, Åarhus, Denmark (1995)Google Scholar
  12. 12.
    Dzidek, W.J., Arisholm, E., Briand, L.C.: A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans. Softw. Eng. 34(3), 407–432 (2008)CrossRefGoogle Scholar
  13. 13.
    Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Inf. Softw. Technol. 52(1), 14–30 (2010)CrossRefGoogle Scholar
  14. 14.
    Farooq, Q.u.a., Iqbal, M.Z.Z., I Malik, Z., Riebisch, M.: A model-based regression testing approach for evolving software systems with flexible tool support. In: Proceedings of the 17th IEEE International Conference and Workshops on Engineering of Computer-Based Systems (ECBS’10), pp. 41–49. IEEE, Oxford, UK (2010)Google Scholar
  15. 15.
    Floch, J., Hallsteinsen, S., Stav, E., Eliassen, F., Lund, K., Gjørven, E.: Beyond design time: using architecture models for runtime adaptability. IEEE Softw. 23(2), 62–70 (2006)CrossRefGoogle Scholar
  16. 16.
    Fowler, M.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Reading (1999)MATHGoogle Scholar
  17. 17.
    France, R.B., Rumpe, B.: Model-Driven Development of Complex Software: A Research Roadmap. In: Briand, L.C., Wolf, A.L. (eds.) Proceedings of Future of Software Engineering (FoSE’07), pp. 37–54. IEEE Computer Society, Minneapolis, MN, USA (2007)Google Scholar
  18. 18.
    Garlan, D., Cheng, S.W., Huang, A.C., Schmerl, B., Steenkiste, P.: Rainbow: architecture-based self adaptation with reusable infrastructure. IEEE Comput. 37(10), 46–54 (2004)CrossRefGoogle Scholar
  19. 19.
    Georgas, J.C., van der Hoek, A., Taylor, R.N.: Using architectural models to manage and visualize runtime adaptation. IEEE Comput. 42(10), 52–60 (2009)CrossRefGoogle Scholar
  20. 20.
    Harrold, M.J.: Testing evolving software. J. Syst. Softw. 47(2–3), 173–181 (1999)CrossRefGoogle Scholar
  21. 21.
    Harrold, M.J., Jones, J.A., Li, T., Liang, D., Orso, A., Pennings, M., Sinha, S., Spoon, S.A., Gujarathi, A.: Regression test selection for Java software. In: Vlissides, J. (ed.) Proceedings of the 16th Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’01), pp. 312–326. ACM, Tampa, FL, USA (2001)Google Scholar
  22. 22.
    Hsia, P., Li, X., Kung, D.C.H., Hsu, C.T., Li, L., Toyoshima, Y., Chen, C.: A technique for the selective revalidation of OO software. J. Softw. Evol. Process 9(4), 217–233 (1997)Google Scholar
  23. 23.
    Korel, B., Tahat, L.H., Vaysburg, B.: Model based regression test reduction using dependence analysis. In: Antoniol, G., Baxter, I.D. (eds.) Proceedings of the International Conference on Software Maintenance (ICSM’02), pp. 214–223. IEEE, Montréal, Quebec, Canada (2002)Google Scholar
  24. 24.
    Kung, D.C., Gao, J., Hsia, P., Toyoshima, Y., Chen, C.: On regression testing of object-oriented programs. J. Syst. Softw. 32(1), 21–40 (1996)CrossRefGoogle Scholar
  25. 25.
    Leung, H.K.N., White, L.J.: Insights into regression testing. In: Proceedings of Conference on Software Maintenance, pp. 60–69. IEEE, Miami, FL, USA (1989)Google Scholar
  26. 26.
    Morin, B., Barais, O., Jézéquel, J.M., Fleurey, F., Solberg, A.: Models@Run.time to support dynamic adaptation. IEEE Comput. 42(10), 44–51 (2009)CrossRefGoogle Scholar
  27. 27.
    Pukall, M., Grebhahn, A., Schröter, R., Kästner, C., Cazzola, W., Götz, S.: JavAdaptor: Unrestricted dynamic software updates for Java. In: Proceedings of the 33rd International Conference on Software Engineering (ICSE’11), pp. 989–991. IEEE, Waikiki, Honolulu, Hawaii (2011)Google Scholar
  28. 28.
    Pukall, M., Kästner, C., Cazzola, W., Götz, S., Grebhahn, A., Schöter, R., Saake, G.: JavAdaptor—Flexible runtime updates of Java applications. Softw.-Pract. Exp. 43(2), 153–185 (2013). https://doi.org/10.1002/spe.2107 CrossRefGoogle Scholar
  29. 29.
    Rothermel, G., Harrold, M.J.: Analyzing regression test selection techniques. IEEE Trans. Softw. Eng. 22(8), 529–551 (1996)CrossRefGoogle Scholar
  30. 30.
    Rothermel, G., Harrold, M.J.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6(2), 173–210 (1997)CrossRefGoogle Scholar
  31. 31.
    Skoglund, M., Runeson, P.: Improving class firewall regression test selection by removing the class firewall. Int. J. Softw. Eng. Knowl. Eng. 17(3), 359–378 (2007)CrossRefGoogle Scholar
  32. 32.
    Soetens, Q.D., Demeyer, S., Zaidman, A.: Change-based test selection in the presence of developer tests. In: Cleve, A., Ricca, F. (eds.) Proceedings of the 17th European Conference on Software Maintenance and Reengineering (CSMR’13), pp. 101–110. IEEE, Genoa, Italy (2013)Google Scholar
  33. 33.
    Soetens, Q.D., Demeyer, S., Zaidman, A., Pérez, J.: Change-based test selection: an empirical evaluation. Empirical Software Engineering, pp. 1–43 (2015)Google Scholar
  34. 34.
    Vogel, T., Giese, H.: Adaptation and abstract runtime models. In: Proceedings of the ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS’10), pp. 39–48. ACM, Cape Town, South Africa (2010)Google Scholar
  35. 35.
    White, L.J., Abdullah, K.: A Firewall approach for regression testing of object-oriented software. In: Proceedings of the 10th International Software Quality Week (QW’97). San Francisco, CA, USA (1997)Google Scholar
  36. 36.
    Yenigün, H.: Identifying the effects of modifications as data dependencies. Softw. Qual. J. 22(4), 701–716 (2014)CrossRefGoogle Scholar
  37. 37.
    Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. J. Softw. Test. Verif. Reliab. 22(2), 67–120 (2012)CrossRefGoogle Scholar
  38. 38.
    Zech, P., Felderer, M., Kalb, P., Breu, R.: A generic platform for model-based regression testing. In: Margaria, T., Steffen, B. (eds.) Proceedings of the 5th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation (ISoLA’12), Lecture Notes in Computer Science 7609, pp. 112–126. Springer, Heraclion, Crete (2012)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  • Mohammed Al-Refai
    • 1
  • Sudipto Ghosh
    • 1
  • Walter Cazzola
    • 2
  1. 1.Computer Science DepartmentColorado State UniversityFort CollinsUSA
  2. 2.Department of Computer ScienceUniversità degli Studi di MilanoMilanItaly

Personalised recommendations