Automated Test Case Selection Using Feature Model: An Industrial Case Study

  • Shuai Wang
  • Arnaud Gotlieb
  • Shaukat Ali
  • Marius Liaaen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8107)


Automated test case selection for a new product in a product line is challenging due to several reasons. First, the variability within the product line needs to be captured in a systematic way; second, the reusable test cases from the repository are required to be identified for testing a new product. The objective of such automated process is to reduce the overall effort for selection (e.g., selection time), while achieving an acceptable level of the coverage of testing functionalities. In this paper, we propose a systematic and automated methodology using a Feature Model for Testing (FM_T) to capture commonalities and variabilities of a product line and a Component Family Model for Testing (CFM_T) to capture the overall structure of test cases in the repository. With our methodology, a test engineer does not need to manually go through the repository to select a relevant set of test cases for a new product. Instead, a test engineer only needs to select a set of relevant features using FM_T at a higher level of abstraction for a product and a set of relevant test cases will be selected automatically. We applied our methodology to a product line of video conferencing systems called Saturn developed by Cisco and the results show that our methodology can reduce the selection effort significantly. Moreover, we conducted a questionnaire-based study to solicit the views of test engineers who were involved in developing FM_T and CFM_T. The results show that test engineers are positive about adapting our methodology and models (FM_T and CFM_T) in their current practice.


Test Case Selection Product Line Feature Model Component Family Model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Benavides, D., Segura, S., Cortés, A.R.: Automated analysis of feature models 20 years later. A literature review. Information Systems (35), 615–636 (2010)Google Scholar
  2. 2.
    Czarnecki, K., Kim, C., Kalleberg, K.: Feature models are views on ontologies. In: Proceedings of the International Software Product Line Conference, pp. 41–51 (2006)Google Scholar
  3. 3.
    Ali, S., Yue, T., Briand, L.C., Walawege, S.: A product line modeling and configuration methodology to support model-based testing: an industrial case study. In: Proceedings of the ACM International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 726–742 (2012)Google Scholar
  4. 4.
    Wang, S., Ali, S., Tao, Y.: Product Line Modeling and Configuration Methodology using Feature Model for Supporting Model-Based Testing. Simula Research Laboratory. Technical Report 2012-24 (2013)Google Scholar
  5. 5.
    McGregor, J.: Testing a Software Product Line. Technical Report. CMU/SEI-2001-TR-022. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania (2001)Google Scholar
  6. 6.
    Engström, E.: Regression Test Selection and Product Line System Testing. In: Proceedings of Third International Conference on Software Testing, Verification and Validation (ICST), pp. 512–515 (2010)Google Scholar
  7. 7.
    Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Information and Software Technology (IST) 52(1), 14–30 (2010)CrossRefGoogle Scholar
  8. 8.
    Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Software: Testing, Verification and Reliability 22(2), 67–120 (2012)CrossRefGoogle Scholar
  9. 9.
  10. 10.
    Cisco Systems: Cisco telepresence codec c90, Data sheet (2010),
  11. 11.
    Wang, S., Gotlieb, A., Liaaen, M., Briand, L.C.: Automatic selection of test execution plans from a Video Conference System Product Line. In: Proceedings of the ACM MODELS Workshop VARiability for You (VARY 2012), pp. 32–37 (2012)Google Scholar
  12. 12.
    Beuche, D., Papajewski, H., Schröder-Preikschat, W.: Variability management with feature models. Science of Computer Programming 53(3), 333–352 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Pure systems GmbH: Variant management with pure:variants. Technical white paper (2006),
  14. 14.
    Pure systems GmbH: Pure:Variants User’s Guide (2011),
  15. 15.
    Wang, S., Ali, S., Gotlieb, A.: Minimizing Test Suites in Software Product Lines Using Weighted-based Genetic Algorithms. Simula Research Laboratory. Technical Report 2012-25 (2013)Google Scholar
  16. 16.
    Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Regnell, B., Wesslen, A.: Experimentation in Software Engineering. Springer (2012)Google Scholar
  17. 17.
    Muccini, H., Van Der Hoek, A.: Towards Testing Product Line Architectures. Electronic Notes in Theoretical Computer Science 82(6), 99–109 (2003)CrossRefGoogle Scholar
  18. 18.
    Uzuncaova, E., Garcia, D., Khurshid, S., Batory, D.: Testing software product lines using incremental test generations. In: Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE), pp. 249–258 (2008)Google Scholar
  19. 19.
    Nebut, C., Le Traon, Y., Jézéquel, J.M.: System Testing of Product Lines: From Requirements to Test Cases. Software Product Lines. In: Research Issues in Engineering and Management, pp. 447–477. Springer (2006)Google Scholar
  20. 20.
    Chen, Y.F., Rosenblum, D.S., Vo, K.P.: Test tube: a system for selective regression testing. In: Proceedings of IEEE International Conference on Software Engineering (ICSE), Los Alamitos, CA, USA, pp. 211–220 (1994)Google Scholar
  21. 21.
    Hartmann, J., Robson, D.J.: Techniques for selective revalidation. IEEE Software 7(1), 31–36 (1990)CrossRefGoogle Scholar
  22. 22.
    Harrold, M.J., Souffa, M.L.: An incremental approach to unit testing during maintenance. In: Proceedings of IEEE International Conference on Software Maintenance (ICSM), pp. 362–367 (1988)Google Scholar
  23. 23.
    Orso, A., Harrold, M.J., Rosenblum, D., Rothermel, G., Soffa, M.L., Do, H.: Using component metacontent to support the regression testing of component-based software. In: Proceedings of IEEE International Conference on Software Maintenance (ICSM), pp. 716–725 (2001)Google Scholar
  24. 24.
    Chen, Y., Probert, R.L., Sims, D.P.: Specification-based regression test selection with risk analysis. In: Proceedings of Conference of the Centre for Advanced Studies on Collaborative Research. IBM Press (2002)Google Scholar
  25. 25.
    Bible, J., Rothermel, G., Rosenblum, D.S.: A comparative study of coarse- and fine- grained safe regression test-selection techniques. ACM Transactions on Software Engineering and Methodology 10(2), 149–183 (2001)CrossRefzbMATHGoogle Scholar
  26. 26.
    Graves, T.L., Harrold, M.J., Kim, J.M., Porter, A., Rothermel, G.: An empirical study of regression test selection techniques. ACM Transactions on Software Engineering and Methodology 10(2), 184–208 (2001)CrossRefzbMATHGoogle Scholar
  27. 27.
    Mansour, N., Bahsoon, R., Baradhi, G.: Empirical comparison of regression test selection algorithms. The Journal of Systems and Software 57(1), 79–90 (2001)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Shuai Wang
    • 1
    • 2
  • Arnaud Gotlieb
    • 1
  • Shaukat Ali
    • 1
  • Marius Liaaen
    • 3
  1. 1.Certus Software V&V Center, Simula Research LaboratoryNorway
  2. 2.Department of InformaticsUniversity of OsloNorway
  3. 3.Cisco Systems Inc.Norway

Personalised recommendations