Skip to main content
SpringerLink
Log in

Supporting inheritance hierarchy changes in model-based regression test selection

  • Theme Section Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

Models can be used to ease and manage the development, evolution, and runtime adaptation of a software system. When models are adapted, the resulting models must be rigorously tested. Apart from adding new test cases, it is also important to perform regression testing to ensure that the evolution or adaptation did not break existing functionality. Since regression testing is performed with limited resources and under time constraints, regression test selection (RTS) techniques are needed to reduce the cost of regression testing. Applying model-level RTS for model-based evolution and adaptation is more convenient than using code-level RTS because the test selection process happens at the same level of abstraction as that of evolution and adaptation. In earlier work, we proposed a model-based RTS approach called MaRTS to be used with a fine-grained model-based adaptation framework that targets applications implemented in Java. MaRTS uses UML models consisting of class and activity diagrams. It classifies test cases as obsolete, reusable, or retestable based on changes made to UML class and activity diagrams of the system being adapted. However, MaRTS did not take into account the changes made to the inheritance hierarchy in the class diagram and the impact of these changes on the selection of test cases. This paper extends MaRTS to support such changes and demonstrates that the extended approach performs as well as or better than code-based RTS approaches in safely selecting regression test cases. While MaRTS can generally be used during any model-driven development or model-based evolution activity, we have developed it in the context of runtime adaptation. We evaluated the extended MaRTS on a set of applications and compared the results with code-based RTS approaches that also support changes to the inheritance hierarchy. The results showed that the extended MaRTS selected all the test cases relevant to the inheritance hierarchy changes and that the fault detection ability of the selected test cases was never lower than that of the baseline test cases. The extended MaRTS achieved comparable results to a graph-walk code-based RTS approach (DejaVu) and showed a higher reduction in the number of selected test cases when compared with a static analysis code-based RTS approach (ChEOPSJ).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://www-03.ibm.com/software/products/en/ratisoftarchsimutool.

  2. https://www.ibm.com/developerworks/rational/library/05/712_comp2/index.html.

  3. http://www.evosuite.org/.

  4. From this point on, the name MaRTS indicates the version that supports changes to the inheritance hierarchy.

  5. http://sofya.unl.edu/doc/manual/user/apps-dejavu.html.

  6. http://win.ua.ac.be/~qsoeten/other/cheopsj/.

  7. http://jung.sourceforge.net/download.html.

  8. http://sir.unl.edu/portal/bios/siena.php.

  9. http://sir.unl.edu/portal/bios/xml-security.php.

  10. http://www.cs.colostate.edu/~malref82/RTSExperiments.zip.

  11. http://pitest.org.

  12. http://pitest.org/quickstart/mutators/.

References

  1. Al-Refai, M., Cazzola, W., Ghosh, S., France, R.: Using models to validate unanticipated, fine-grained adaptations at runtime. In: Waeselynck, H., Babiceanu, R.(eds.) Proceedings of the 17th IEEE International Symposium on High Assurance Systems Engineering (HASE’16), pp. 23–30. IEEE, Orlando, FL, USA (2016)

  2. Al-Refai, M., Ghosh, S., Cazzola, W.: Model-based regression test selection for validating runtime adaptation of software systems. In: Briand, L., Khurshid, S. (eds.) Proceedings of the 9th IEEE international conference on software testing, verification and validation (ICST’16), pp. 288–298. IEEE, Chicago, IL, USA (2016)

  3. Arcuri, A., Campos, J., Fraser, G.: Unit test generation during software development: EvoSuite Plugins for Maven, IntelliJ and Jenkins. In: Briand, L., Khurshid, S. (eds.) Proceedings of the 9th IEEE International Conference on Software Testing, Verification and Validation (ICST’16), pp. 401–408. IEEE, Chicago, IL, USA (2016)

  4. Briand, L.C., Labiche, Y., He, S.: Automating regression test selection based on UML designs. J. Inf. Softw. Technol. 51(1), 16–30 (2009)

    Article  Google Scholar 

  5. Cazzola, W., Pini, S., Ghoneim, A., Saake, G.: Co-evolving application code and design models by exploiting meta-data. In: Proceedings of the 22nd Annual ACM Symposium on Applied Computing (SAC’07), pp. 1275–1279. ACM Press, Seoul, South Korea (2007)

  6. Cazzola, W., Rossini, N.A., Al-Refai, M., France, R.B.: Fine-grained software evolution using UML activity and class models. In: Moreira, A., Schätz, B. (eds.) Proceedings of the 16th International Conference on Model Driven Engineering Languages and Systems (MoDELS’13), Lecture Notes in Computer Science 8107, pp. 271–286. Springer, Miami, FL, USA (2013)

  7. Cazzola, W., Rossini, N.A., Bennett, P., Pradeep Mandalaparty, S., France, R.B.: Fine-grained semi-automated runtime evolution. In: Bencomo, N., Chang, B., France, R.B., Aßmann, U. (eds.) MoDELS@Run-Time, Lecture Notes in Computer Science 8378, pp. 237–258. Springer (2014)

  8. Cazzola, W., Vacchi, E.: @Java: bringing a richer annotation model to Java. Comput. Lang. Syst. Struct. 40(1), 2–18 (2014). https://doi.org/10.1016/j.cl.2014.02.002

    Google Scholar 

  9. Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Stewart, D.A., Johnson, J.H. (eds.) Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research (CASCON’02), pp. 1–14. IBM Press (2002)

  10. Cottenier, T., van den Berg, A., Elrad, T.: Motorola WEAVR: aspect orientation and model-driven engineering. J. Obj. Technol. 6(7), 51–88 (2007)

    Article  Google Scholar 

  11. Dean, J., Grove, D., Chambers, C.: Optimization of object-oriented programs using static class hierarchy analysis. In: Olthoff, W.G. (ed.) Proceedings of the 9th European Conference on Object-Oriented Programming (ECOOP’95), LNCS 952, pp. 77–101. Springer, Åarhus, Denmark (1995)

  12. Dzidek, W.J., Arisholm, E., Briand, L.C.: A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans. Softw. Eng. 34(3), 407–432 (2008)

    Article  Google Scholar 

  13. Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Inf. Softw. Technol. 52(1), 14–30 (2010)

    Article  Google Scholar 

  14. Farooq, Q.u.a., Iqbal, M.Z.Z., I Malik, Z., Riebisch, M.: A model-based regression testing approach for evolving software systems with flexible tool support. In: Proceedings of the 17th IEEE International Conference and Workshops on Engineering of Computer-Based Systems (ECBS’10), pp. 41–49. IEEE, Oxford, UK (2010)

  15. Floch, J., Hallsteinsen, S., Stav, E., Eliassen, F., Lund, K., Gjørven, E.: Beyond design time: using architecture models for runtime adaptability. IEEE Softw. 23(2), 62–70 (2006)

    Article  Google Scholar 

  16. Fowler, M.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Reading (1999)

    MATH  Google Scholar 

  17. France, R.B., Rumpe, B.: Model-Driven Development of Complex Software: A Research Roadmap. In: Briand, L.C., Wolf, A.L. (eds.) Proceedings of Future of Software Engineering (FoSE’07), pp. 37–54. IEEE Computer Society, Minneapolis, MN, USA (2007)

  18. Garlan, D., Cheng, S.W., Huang, A.C., Schmerl, B., Steenkiste, P.: Rainbow: architecture-based self adaptation with reusable infrastructure. IEEE Comput. 37(10), 46–54 (2004)

    Article  Google Scholar 

  19. Georgas, J.C., van der Hoek, A., Taylor, R.N.: Using architectural models to manage and visualize runtime adaptation. IEEE Comput. 42(10), 52–60 (2009)

    Article  Google Scholar 

  20. Harrold, M.J.: Testing evolving software. J. Syst. Softw. 47(2–3), 173–181 (1999)

    Article  Google Scholar 

  21. Harrold, M.J., Jones, J.A., Li, T., Liang, D., Orso, A., Pennings, M., Sinha, S., Spoon, S.A., Gujarathi, A.: Regression test selection for Java software. In: Vlissides, J. (ed.) Proceedings of the 16th Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’01), pp. 312–326. ACM, Tampa, FL, USA (2001)

  22. Hsia, P., Li, X., Kung, D.C.H., Hsu, C.T., Li, L., Toyoshima, Y., Chen, C.: A technique for the selective revalidation of OO software. J. Softw. Evol. Process 9(4), 217–233 (1997)

    Google Scholar 

  23. Korel, B., Tahat, L.H., Vaysburg, B.: Model based regression test reduction using dependence analysis. In: Antoniol, G., Baxter, I.D. (eds.) Proceedings of the International Conference on Software Maintenance (ICSM’02), pp. 214–223. IEEE, Montréal, Quebec, Canada (2002)

  24. Kung, D.C., Gao, J., Hsia, P., Toyoshima, Y., Chen, C.: On regression testing of object-oriented programs. J. Syst. Softw. 32(1), 21–40 (1996)

    Article  Google Scholar 

  25. Leung, H.K.N., White, L.J.: Insights into regression testing. In: Proceedings of Conference on Software Maintenance, pp. 60–69. IEEE, Miami, FL, USA (1989)

  26. Morin, B., Barais, O., Jézéquel, J.M., Fleurey, F., Solberg, A.: Models@Run.time to support dynamic adaptation. IEEE Comput. 42(10), 44–51 (2009)

    Article  Google Scholar 

  27. Pukall, M., Grebhahn, A., Schröter, R., Kästner, C., Cazzola, W., Götz, S.: JavAdaptor: Unrestricted dynamic software updates for Java. In: Proceedings of the 33rd International Conference on Software Engineering (ICSE’11), pp. 989–991. IEEE, Waikiki, Honolulu, Hawaii (2011)

  28. Pukall, M., Kästner, C., Cazzola, W., Götz, S., Grebhahn, A., Schöter, R., Saake, G.: JavAdaptor—Flexible runtime updates of Java applications. Softw.-Pract. Exp. 43(2), 153–185 (2013). https://doi.org/10.1002/spe.2107

    Article  Google Scholar 

  29. Rothermel, G., Harrold, M.J.: Analyzing regression test selection techniques. IEEE Trans. Softw. Eng. 22(8), 529–551 (1996)

    Article  Google Scholar 

  30. Rothermel, G., Harrold, M.J.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6(2), 173–210 (1997)

    Article  Google Scholar 

  31. Skoglund, M., Runeson, P.: Improving class firewall regression test selection by removing the class firewall. Int. J. Softw. Eng. Knowl. Eng. 17(3), 359–378 (2007)

    Article  Google Scholar 

  32. Soetens, Q.D., Demeyer, S., Zaidman, A.: Change-based test selection in the presence of developer tests. In: Cleve, A., Ricca, F. (eds.) Proceedings of the 17th European Conference on Software Maintenance and Reengineering (CSMR’13), pp. 101–110. IEEE, Genoa, Italy (2013)

  33. Soetens, Q.D., Demeyer, S., Zaidman, A., Pérez, J.: Change-based test selection: an empirical evaluation. Empirical Software Engineering, pp. 1–43 (2015)

  34. Vogel, T., Giese, H.: Adaptation and abstract runtime models. In: Proceedings of the ICSE Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS’10), pp. 39–48. ACM, Cape Town, South Africa (2010)

  35. White, L.J., Abdullah, K.: A Firewall approach for regression testing of object-oriented software. In: Proceedings of the 10th International Software Quality Week (QW’97). San Francisco, CA, USA (1997)

  36. Yenigün, H.: Identifying the effects of modifications as data dependencies. Softw. Qual. J. 22(4), 701–716 (2014)

    Article  Google Scholar 

  37. Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. J. Softw. Test. Verif. Reliab. 22(2), 67–120 (2012)

    Article  Google Scholar 

  38. Zech, P., Felderer, M., Kalb, P., Breu, R.: A generic platform for model-based regression testing. In: Margaria, T., Steffen, B. (eds.) Proceedings of the 5th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation (ISoLA’12), Lecture Notes in Computer Science 7609, pp. 112–126. Springer, Heraclion, Crete (2012)

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. CNS 1305381. We would like to acknowledge the support of Quinten Soetens who helped us install and run ChEOPSJ.

Author information

Authors and Affiliations

  1. Computer Science Department, Colorado State University, Fort Collins, CO, USA

    Mohammed Al-Refai & Sudipto Ghosh

  2. Department of Computer Science, Università degli Studi di Milano, Milan, Italy

    Walter Cazzola

Authors
  1. Mohammed Al-Refai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudipto Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Walter Cazzola
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammed Al-Refai.

Additional information

Communicated by Dr . M. Papadakis, S. Ali, and G. Perrouin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Al-Refai, M., Ghosh, S. & Cazzola, W. Supporting inheritance hierarchy changes in model-based regression test selection. Softw Syst Model 18, 937–958 (2019). https://doi.org/10.1007/s10270-017-0636-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-017-0636-3

Keywords

Search

Navigation