Evolutionary Search-Based Test Generation for Software Product Line Feature Models

  • Faezeh Ensan
  • Ebrahim Bagheri
  • Dragan Gašević
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7328)


Product line-based software engineering is a paradigm that models the commonalities and variabilities of different applications of a given domain of interest within a unique framework and enhances rapid and low cost development of new applications based on reuse engineering principles. Despite the numerous advantages of software product lines, it is quite challenging to comprehensively test them. This is due to the fact that a product line can potentially represent many different applications; therefore, testing a single product line requires the test of its various applications. Theoretically, a product line with n software features can be a source for the development of 2 n application. This requires the test of 2 n applications if a brute-force comprehensive testing strategy is adopted. In this paper, we propose an evolutionary testing approach based on Genetic Algorithms to explore the configuration space of a software product line feature model in order to automatically generate test suites. We will show through the use of several publicly-available product line feature models that the proposed approach is able to generate test suites of O(n) size complexity as opposed to O(2 n ) while at the same time form a suitable tradeoff balance between error coverage and feature coverage in its generated test suites.


Software product lines Feature models Evolutionary testing 


  1. 1.
    Bagheri, E., Deldari, H.: Dejong function optimization by means of a parallel approach to fuzzified genetic algorithm. In: IEEE Symposium on Computers and Communications, pp. 675–680 (2006)Google Scholar
  2. 2.
    Bagheri, E., Gasevic, D.: Assessing the maintainability of software product line feature models using structural metrics. Software Quality Journal 19(3), 579–612 (2011)CrossRefGoogle Scholar
  3. 3.
    Benavides, D., Segura, S., Trinidad, P., Ruiz-Cortes, A.: FAMA: Tooling a framework for the automated analysis of feature models. In: Proceeding of the First International Workshop on Variability Modelling of Software-intensive Systems (VAMOS), pp. 129–134 (2007)Google Scholar
  4. 4.
    Buehler, O., Wegener, J.: Evolutionary functional testing of an automated parking system. In: International Conference on Computer, Communication and Control Technologies (CCCT 2003). Citeseer (2003)Google Scholar
  5. 5.
    Clements, P., Northrop, L.M.: Software product lines (2003), (visited June 2009)
  6. 6.
    Coello, C., Lamont, G., Van Veldhuizen, D.: Evolutionary algorithms for solving multi-objective problems. Springer-Verlag New York Inc. (2007)Google Scholar
  7. 7.
    Cohen, M., Dwyer, M., Shi, J.: Coverage and adequacy in software product line testing. In: Proceedings of the ISSTA 2006 Workshop on Role of Software Architecture for Testing and Analysis, p. 63. ACM (2006)Google Scholar
  8. 8.
    Cohen, M.B., Dwyer, M.B., Shi, J.: Coverage and adequacy in software product line testing. In: Proceedings of the ISSTA 2006 Workshop on Role of Software Architecture for Testing and Analysis, ROSATEA 2006, pp. 53–63. ACM, New York (2006)CrossRefGoogle Scholar
  9. 9.
    Dallal, J., Sorenson, P.: Testing software assets of framework-based product families during application engineering stage. Journal of Software 3(5), 11 (2008)CrossRefGoogle Scholar
  10. 10.
    Goldberg, D.: Genetic Algorithms in Search and Optimization (1989)Google Scholar
  11. 11.
    Kang, K., Lee, J., Donohoe, P.: Feature-oriented product line engineering. IEEE Software 19(4), 58–65 (2002)CrossRefGoogle Scholar
  12. 12.
    Khurshid, S., Marinov, D.: TestEra: Specification-based testing of Java programs using SAT. Automated Software Engineering 11(4), 403–434 (2004)CrossRefGoogle Scholar
  13. 13.
    Kim, C., Batory, D., Khurshid, S.: Reducing Combinatorics in Testing Product Lines (Technical Report), UTexas (2010)Google Scholar
  14. 14.
    Lee, K., Kang, K., Lee, J.: Concepts and Guidelines of Feature Modeling for Product Line Software Engineering. In: Gacek, C. (ed.) ICSR 2002. LNCS, vol. 2319, pp. 62–77. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    McMinn, P.: Search-based software test data generation: A survey. Software Testing, Verification and Reliability 14(2), 105–156 (2004)CrossRefGoogle Scholar
  16. 16.
    Meffert, K., Rotstan, N., Knowles, C., Sangiorgi, U.: JGAP–Java Genetic Algorithms and Genetic Programming Package,
  17. 17.
    Mendonca, M., Wasowski, A., Czarnecki, K., Cowan, D.: Efficient compilation techniques for large scale feature models. In: Proceedings of the 7th International Conference on Generative Programming and Component Engineering, pp. 13–22. ACM, New York (2008)CrossRefGoogle Scholar
  18. 18.
    Mendonca, M., Branco, M., Cowan, D.: S.p.l.o.t.: software product lines online tools. In: OOPSLA 2009: Proceeding of the 24th ACM SIGPLAN Conference Companion on Object Oriented Programming Systems Languages and Applications, pp. 761–762 (2009)Google Scholar
  19. 19.
    Nebut, C., Fleurey, F., Le Traon, Y., Jézéquel, J.-M.: A Requirement-Based Approach to Test Product Families. In: van der Linden, F.J. (ed.) PFE 2003. LNCS, vol. 3014, pp. 198–210. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  20. 20.
    Olimpiew, E., Gomaa, H.: Model-based testing for applications derived from software product lines. In: Proceedings of the 1st International Workshop on Advances in Model-based Testing, p. 7. ACM (2005)Google Scholar
  21. 21.
    Perrouin, G., Sen, S., Klein, J., Baudry, B., Le Traon, Y.: Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines. In: ICST 2010, pp. 459–468 (2010)Google Scholar
  22. 22.
    Pohl, K., Böckle, G., Van Der Linden, F.: Software Product Line Engineering: Foundations, Principles, and Techniques. Springer (2005)Google Scholar
  23. 23.
    Pohl, K., Metzger, A.: Software product line testing. Communications of the ACM 49(12), 81 (2006)CrossRefGoogle Scholar
  24. 24.
    Reis, S., Metzger, A., Pohl, K.: Integration Testing in Software Product Line Engineering: A Model-Based Technique. In: Dwyer, M.B., Lopes, A. (eds.) FASE 2007. LNCS, vol. 4422, pp. 321–335. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  25. 25.
    Scheidemann, K.: Optimizing the selection of representative configurations in verification of evolving product lines of distributed embedded systems, SPLC 2006 (2006)Google Scholar
  26. 26.
    Segura, S., Hierons, R., Benavides, D., Ruiz-Cortés, A.: Automated test data generation on the analyses of feature models: A metamorphic testing approach. In: ICST 2010, pp. 35–44 (2010)Google Scholar
  27. 27.
    Tracey, N., Clark, J., Mander, K.: Automated program flaw finding using simulated annealing. In: Proceedings of the 1998 ACM SIGSOFT International Symposium on Software Testing and Analysis, p. 81. ACM (1998)Google Scholar
  28. 28.
    Uzuncaova, E., Garcia, D., Khurshid, S., Batory, D.S.: A specification-based approach to testing software product lines. In: ESEC/SIGSOFT FSE, pp. 525–528 (2007)Google Scholar
  29. 29.
    Uzuncaova, E., Khurshid, S., Batory, D.S.: Incremental test generation for software product lines. IEEE Trans. Software Eng. 36(3), 309–322 (2010)CrossRefGoogle Scholar
  30. 30.
    Watkins, A.: The automatic generation of test data using genetic algorithms. In: Proceedings of the 4th Software Quality Conference, vol. 2, pp. 300–309 (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Faezeh Ensan
    • 1
  • Ebrahim Bagheri
    • 2
  • Dragan Gašević
    • 2
  1. 1.University of British ColumbiaCanada
  2. 2.Athabasca UniversityCanada

Personalised recommendations