Automated Incremental Pairwise Testing of Software Product Lines

  • Sebastian Oster
  • Florian Markert
  • Philipp Ritter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6287)


Testing Software Product Lines is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today’s testing approaches for Software Product Lines validate products individually using different kinds of reuse techniques for testing. Due to the enormous number of possible products, individual product testing becomes more and more unfeasible. Combinatorial testing offers one possibility to test a subset of all possible products. In this contribution we provide a detailed description of a methodology to apply combinatorial testing to a feature model of a Software Product Line. We combine graph transformation, combinatorial testing, and forward checking for that purpose. Additionally, our approach considers predefined sets of products.


Software Product Line Pairwise Testing Combinatorial Testing Software Product Line Engineering Valid Product 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Clements, P., Northrop, L.: Software product lines: practices and patterns. Addison-Wesley Longman Publishing Co., Inc., Boston (2001)Google Scholar
  2. 2.
    Pohl, K., Böckle, G., Van der Linden, F.J.: Software Product Line Engineering: Foundations, Principles and Techniques. Springer, New York (2005)zbMATHGoogle Scholar
  3. 3.
    Czarnecki, K., Eisenecker, U.: Generative Programming: Methods, Tools, and Applications. Addison-Wesley Professional, Reading (June 2000)Google Scholar
  4. 4.
    Bosch, J.: Design and Use of Software Architectures - Adopting and Evolving a Product Line Approach (2000)Google Scholar
  5. 5.
    Czarnecki, K., Helsen, S., Eisenecker, U.: Staged configuration through specialization and multilevel configuration of feature models. Software Process: Improvement and Practice 10(2), 143–169 (2005)CrossRefGoogle Scholar
  6. 6.
    Kang, K.C., Cohen, S.G., Hess, J.A., Novak, W.E., Peterson, A.S.: Feature-oriented domain analysis (foda) feasibility study. Technical report, Carnegie-Mellon University Software Engineering Institute (November 1990)Google Scholar
  7. 7.
    Heymans, P., Schobbens, P.Y., Trigaux, J.C., Bontemps, Y., Matulevicius, R., Classen, A.: Evaluating formal properties of feature diagram languages. Software, IET 2(3), 281–302 (2008)CrossRefGoogle Scholar
  8. 8.
  9. 9.
    Beizer, B.: Software testing techniques, 2nd edn. Van Nostrand Reinhold Co., New York (1990)Google Scholar
  10. 10.
    Cohen, M., Dwyer, M., Shi, J.: Interaction testing of highly-configurable systems in the presence of constraints. In: ISSTA, pp. 129–139 (2007)Google Scholar
  11. 11.
    Stevens, B., Mendelsohn, E.: Efficient software testing protocols. In: Conference of the Centre for Advanced Studies on Collaborative Research. IBM Press (1998)Google Scholar
  12. 12.
    Cohen, D.M., Dalal, S.R., Kajla, A., Patton, G.: The automatic efficient tests generator. In: Fifth ISSRE IEEE, pp. 303–309 (1994)Google Scholar
  13. 13.
    Lei, Y., Tai, K.: In-parameter-order: a test generation strategy for pairwise testing. In: IEEE High Assurance Systems Engineering Symposium, pp. 254–261 (1998)Google Scholar
  14. 14.
    feasiPLE Consortium (2006-2009),
  15. 15.
    Wübbeke, A.: Towards an Efficient Reuse of Test Cases for Software Product Lines. In: Thiel, S., Pohl, K. (eds.) Proceedings of the 12th International Software Product Line Conference Second Volume, pp. 361–368 (2008)Google Scholar
  16. 16.
    Weißleder, S., Sokenou, D., Schlinglo, B.: Reusing State Machines for Automatic Test Generation in Product Lines. In: Proceedings of the 1st Workshop on Model-based Testing in Practice (MoTiP 2008) (2008)Google Scholar
  17. 17.
    Reuys, A., Kamsties, E., Pohl, K., Reis, S.: Model-based System Testing of Software Product Families. In: Pastor, Ó., Falcão e Cunha, J. (eds.) CAiSE 2005. LNCS, vol. 3520, pp. 519–534. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  18. 18.
    Olimpiew, E.M.: Model-Based Testing for Software Product Lines. PhD thesis, George Mason University (2008)Google Scholar
  19. 19.
    Oster, S., Wübbeke, A., Engels, G., Schürr, A.: Model-Based Software Product Lines Testing Survey. In: Zander, J., Schieferdecker, I., Mosterman, P. (eds.) Model-based Testing for Embedded Systems. CRC Press/Taylor&Francis (to appear, 2010)Google Scholar
  20. 20.
    Haralick, R., Elliott, G.: Increasing tree search efficiency for constraint satisfaction problems. Artificial intelligence 14(3), 263–313 (1980)CrossRefGoogle Scholar
  21. 21.
    Perrouin, G., Sen, S., Klein, J., Baudry, B., Traon, Y.L.: Automated and scalable t-wise test case generation strategies for software product lines. In: Third International Conference on Software Testing, Verification and Validation (2010)Google Scholar
  22. 22.
    Oster, S., Markert, F., Schürr, A.: Integrated Modeling of Software Product Lines with Feature Models and Classification Trees. In: Proceedings of the 13th International Software Product Line Conference (SPLC 2009). MAPLE 2009 Workshop Proceedings. Springer, Heidelberg (2009)Google Scholar
  23. 23.
    Schürr, A., Oster, S., Markert, F.: Model-Driven Software Product Line Testing: An Integrated Approach. In: 36th International Conference on Current Trends in Theory and Practice of Computer Science. LNCS, pp. 112–131. Springer, Heidelberg (2009)Google Scholar
  24. 24.
    Bennaceur, H.: A Comparison between SAT and CSP Techniques. Constraints 9(2), 123–138 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Westphal, M., Wölfl, S.: Qualitative csp, finite csp, and sat: comparing methods for qualitative constraint-based reasoning. In: IJCAI 2009: Proceedings of the 21st international jont conference on Artifical intelligence, pp. 628–633. Morgan Kaufmann Publishers Inc., San Francisco (2009)Google Scholar
  26. 26.
    Oster, S., Schürr, A., Weisemöller, I.: Towards Software Product Line Testing using Story Driven Modelling. In: Aßmann, U., Johannes, J., Zündorf, A. (eds.) Proceedings of the 6th Int. Fujaba Days, TU Dresden, pp. 48–51 (2008)Google Scholar
  27. 27.
    Oster, S., Ritter, P., Schürr, A.: Featuremodellbasiertes und kombinatorisches Testen von Software-Produktlinien. In: Proceedings of the SE 2010. GI-Edition Lecture Notes in Informatics. Gesellschaft für Informatik (2010)Google Scholar
  28. 28.
  29. 29.
    Thum, T., Batory, D., Kastner, C.: Reasoning about edits to feature models. In: ICSE 2009: Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, pp. 254–264. IEEE Computer Society, Washington (2009)CrossRefGoogle Scholar
  30. 30.
    Tevanlinna, A., Taina, J., Kauppinen, R.: Product family testing: a survey. ACM SIGSOFT Software Engineering Notes 29, 12 (2004)CrossRefGoogle Scholar
  31. 31.
    Scheidemann, K.: Verifying families of system configurations. Doctoral Thesis TU Munich (2007)Google Scholar
  32. 32.
    McGregor, J.D.: Testing a software product line. Technical Report CMU/SEI-2001-TR-022 (2001)Google Scholar
  33. 33.
    Cohen, M.B., Dwyer, M.B., Shi, J.: Coverage and adequacy in software product line testing. In: ROSATEA 2006: Proceedings of the ISSTA 2006 workshop, pp. 53–63. ACM, New York (2006)Google Scholar
  34. 34.
    White, J., Dougherty, B., Schmidt, D.C.: Selecting highly optimal architectural feature sets with filtered cartesian flattening. Journal of Systems and Software 82(8), 1268–1284 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sebastian Oster
    • 1
  • Florian Markert
    • 2
  • Philipp Ritter
    • 1
  1. 1.Real-Time Systems Group 
  2. 2.Computer Systems GroupTechnische Universität DarmstadtGermany

Personalised recommendations