Automated Test Design for Boundaries of Product Line Variants

  • Stephan Weißleder
  • Florian Wartenberg
  • Hartmut LacknerEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9447)


Developing product lines is usually more efficient than developing single products because of the reuse of single components. Testing, however, has to consider complete, integrated systems. To prevent testing every product on system level, the whole product line should be analyzed with the aim of selecting distinguishing product behavior and a minimum of system products to test. In this paper, we present a model-based test design approach for testing the selected behavior of products, but also their deselected behavior. A major challenge of this approach is that the deselected behavior of a product is often not part of its behavioral model. Thus, we use the variability model to transform the behavioral model so that showing the exclusion of the deselected behavior is also covered by tests. We present the approach, a corresponding prototypical implementation, and our experiences using a set of examples.


Unify Modeling Language Test Suite Software Product Line Boundary Transition Complementary Transition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work is partially supported by grants from Deutsche Forschungsgemeinschaft, Graduiertenkolleg METRIK (GRK 1324).


  1. 1.
    Andrews, J.H., Briand, L.C., Labiche, Y.: Is mutation an appropriate tool for testing experiments? In: Proceedings of the 27th International Conference on Software Engineering, ICSE 2005, pp. 402–411 (2005)Google Scholar
  2. 2.
    Andrews, J.H., Briand, L.C., Labiche, Y., Namin, A.S.: Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Trans. Softw. Eng. 32(8), 608–624 (2006)CrossRefGoogle Scholar
  3. 3.
    Binder, R.V.: Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)Google Scholar
  4. 4.
    Briand, L.C., Penta, M.D., Labiche, Y.: Assessing and improving state-based class testing: a series of experiments. IEEE Trans. Softw. Eng. 30(11), 770–783 (2004)CrossRefGoogle Scholar
  5. 5.
    Broy, M., Jonsson, B., Katoen, J.P.: Model-Based Testing of Reactive Systems: Advanced Lectures. Lecture Notes in Computer Science, vol. 3472. Springer, Heidelberg (2005)Google Scholar
  6. 6.
    Cichos, H., Heinze, T.S.: Efficient reduction of model-based generated test suites through test case pair prioritization. In: Proceedings of the 7th International Workshop on Model-Driven Engineering. Verification and Validation (MoDeVVa 10), pp. 37–42. IEEE Computer Society Press, Los Alamitos (2011)Google Scholar
  7. 7.
    Cichos, H., Lochau, M., Oster, S., Schürr, A.: Reduktion von testsuiten für software-produktlinien. In: Jähnichen, S., Küpper, A., Albayrak, S. (eds.) Software Engineering 2012: Fachtagung des GI-Fachbereichs Softwaretechnik, 27. Februar - 2. März 2012 in Berlin. LNI, vol. 198, pp. 143–154. GI (2012)Google Scholar
  8. 8.
    Classen, A., Heymans, P., Schobbens, P.Y., Legay, A.: Symbolic model checking of software product lines. In: 33rd International Conference on Software Engineering, ICSE 2011, May 21–28, 2011, Waikiki, Honolulu, Hawaii, Proceedings, pp. 321–330. ACM (2011)Google Scholar
  9. 9.
    Conformiq Qtronic: Semantics and Algorithms for Test Generation: A Conformiq Software Whitepaper (2008)Google Scholar
  10. 10.
    Czarnecki, K., Antkiewicz, M.: Mapping features to models: a template approach based on superimposed variants. In: Glück, R., Lowry, M. (eds.) GPCE 2005. LNCS, vol. 3676, pp. 422–437. Springer, Heidelberg (2005) CrossRefGoogle Scholar
  11. 11.
    Czarnecki, K., Wasowski, A.: Feature diagrams and logics: there andback again. In: Software Product Line Conference, 2007. SPLC 2007. 11th International, pp. 23–34 (2007)Google Scholar
  12. 12.
    DeMillo, R.A.: Mutation Analysis as a Tool for Software Quality Assurance. In: COMPSAC 1980 (1980)Google Scholar
  13. 13.
    Grönniger, H., Krahn, H., Pinkernell, C., Rumpe, B.: Modeling variants of automotive systems using views. In: Kühne, T., Reisig, W., Steimann, F. (eds.) Tagungsband zur Modellierung 2008 (Berlin-Adlershof, Deutschland, 12–14. März 2008). LNI, Gesellschaft für Informatik, Bonn (2008)Google Scholar
  14. 14.
    Holt, N.E., Torkar, R., Briand, L.C., Hansen, K.: State-based testing: Industrial evaluation of the cost-effectiveness of round-trip path and sneak-path strategies. In: 23rd IEEE International Symposium on Software Reliability Engineering, ISSRE 2012, Dallas, TX, USA, November 27–30, pp. 321–330. IEEE Computer Society (2012)Google Scholar
  15. 15.
    Kang, K.C., Cohen, S.G., Hess, J.A., Novak, W.E., Peterson, A.S.: Feature-Oriented Domain Analysis (FODA) Feasibility Study (1990)Google Scholar
  16. 16.
    Lackner, H., Thomas, M., Wartenberg, F., Weißleder, S.: Model-based test design of product lines: raising test design to the product line level. In: ICST 2014: International Conference on Software Testing, Verification, and Validation, pp. 51–60. IEEE Computer Society (2014)Google Scholar
  17. 17.
    Mouchawrab, S., Briand, L.C., Labiche, Y., Di Penta, M.: Assessing, comparing, and combining state machine-based testing and structural testing: a series of experiments. IEEE Trans. Softw. Eng. 37(2), 161–187 (2011)CrossRefGoogle Scholar
  18. 18.
    Oster, S., Wubbeke, A., Engels, G., Schürr, A.: A survey of model-based software product lines testing. In: Zander, J., Schieferdecker, I., Mosterman, P.J. (eds.) Model-Based Testing for Embedded Systems. Computational Analysis, Synthesis, and Design of Dynamic Systems, pp. 339–384. CRC Press, Boca Raton (2011)Google Scholar
  19. 19.
    Paradkar, A.: Case studies on fault detection effectiveness of model based test generation techniques. In: Proceedings of the 1st International Workshop on Advances in Model-based Testing, A-MOST 2005, pp. 1–7 (2005)Google Scholar
  20. 20.
    Peleska, J.: RT-Tester Model-Based Test Case and Test Data Generator: User Manual: Version 9.0–1.0.0 (2013)Google Scholar
  21. 21.
    Pohl, K., Böckle, G., van der Linden, F.: Software Product Line Engineering: Foundations, Principles and Techniques. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  22. 22.
    Siami Namin, A., Andrews, J.H., Murdoch, D.J.: Sufficient mutation operators for measuring test effectiveness. In: Proceedings of the 30th International Conference on Software Engineering, ICSE 2008, pp. 351–360 (2008)Google Scholar
  23. 23.
    Smith, B.H., Williams, L.: Should software testers use mutation analysis to augment a test set? J. Syst. Softw. 82(11), 1819–1832 (2009)CrossRefGoogle Scholar
  24. 24.
    Weißleder, S., Schlingloff, H.: An evaluation of model-based testing in embedded applications. In: ICST 2014: International Conference on Software Testing, Verification, and Validation. IEEE Computer Society (2014)Google Scholar
  25. 25.
    Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach, 1st edn. Morgan Kaufmann Publishers Inc., San Francisco (2006)Google Scholar
  26. 26.
    Weißleder, S.: Simulated satisfaction of coverage criteria on UML state machines. In: ICST - 3rd International Conference on Software Testing, Verification and Validation (2010)Google Scholar
  27. 27.
    Weißleder, S.: ParTeG (Partition Test Generator) (2009)Google Scholar
  28. 28.
    Weißleder, S., Schlingloff, H.: Automatic model-based test generation from UML state machines. In: Zander, J., Schieferdecker, I., Mosterman, P.J. (eds.) Model-Based Testing for Embedded Systems. Computational Analysis, Synthesis, and Design of Dynamic Systems. CRC Press, Boca Raton (2011)Google Scholar
  29. 29.
    Zander, J., Schieferdecker, I., Mosterman, P.J.: A taxonomy of model-based testing for embedded systems from multiple industry domains. In: Zander, J., Schieferdecker, I., Mosterman, P.J. (eds.) Model-based testing for embedded systems. Computational Analysis, Synthesis, and Design of Dynamic Systems. CRC Press, Boca Raton (2011)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2015

Authors and Affiliations

  • Stephan Weißleder
    • 1
  • Florian Wartenberg
    • 1
  • Hartmut Lackner
    • 2
    Email author
  1. 1.Thales Transportation SystemsBerlinGermany
  2. 2.Humboldt-Universität zu BerlinBerlinGermany

Personalised recommendations