Coverage Criteria for Behavioural Testing of Software Product Lines

  • Xavier Devroey
  • Gilles Perrouin
  • Axel Legay
  • Maxime Cordy
  • Pierre-Yves Schobbens
  • Patrick Heymans
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8802)


Featured Transition Systems (FTS) is a mathematical structure to represent the behaviour of software product line in a concise way. The combination of the well-known transition systems approach to formal behavioural modelling with feature expressions was pivotal to the design of efficient verification approaches. Such approaches indeed avoid to consider products’ behaviour independently, leading to often exponential savings. Building on this successful structure, we lay the foundations of model-based testing approach to SPLs. We define several FTS-aware coverage criteria and report on our experience combining FTS with usage-based testing for configurable websites.


Coverage Criteria Model Based Testing Software Product Line Engineering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    ALL4TEC: MaTeLo, (last visit January 31, 2014)
  2. 2.
    Astesana, J.M., Cosserat, L., Fargier, H.: Constraint-based vehicle configuration: A case study. In: 2010 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI), vol. 1, pp. 68–75 (October 2010)Google Scholar
  3. 3.
    Beohar, H., Mousavi, M.R.: Spinal Test Suites for Software Product Lines. ArXiv e-prints (2014)Google Scholar
  4. 4.
    Cichos, H., Oster, S., Lochau, M., Schürr, A.: Model-based Coverage-driven Test Suite Generation for Software Product Lines. In: Whittle, J., Clark, T., Kühne, T. (eds.) MODELS 2011. LNCS, vol. 6981, pp. 425–439. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Classen, A.: Modelling and Model Checking Variability-Intensive Systems. Ph.D. thesis, PReCISE Research Center, Faculty of Computer Science, University of Namur, FUNDP (2011)Google Scholar
  6. 6.
    Classen, A., Cordy, M., Schobbens, P.Y., Heymans, P., Legay, A., Raskin, J.F.: Featured Transition Systems: Foundations for Verifying Variability-Intensive Systems and their Application to LTL Model Checking. TSE PP(99), 1–22 (2013)Google Scholar
  7. 7.
    Classen, A., Heymans, P., Schobbens, P.Y., Legay, A., Raskin, J.F.: Model checking lots of systems: efficient verification of temporal properties in software product lines. In: ICSE 2010, pp. 335–344. ACM (2010)Google Scholar
  8. 8.
    Clements, P., Northrop, L.: Software Product Lines: Practices and Patterns. Addison Wesley, Reading (2001)Google Scholar
  9. 9.
    Cohen, M., Dwyer, M., Shi, J.: Interaction testing of highly-configurable systems in the presence of constraints. In: ISSTA 2007, pp. 129–139 (2007)Google Scholar
  10. 10.
    Cordy, M., Classen, A., Heymans, P., Schobbens, P.Y., Legay, A.: Provelines: A product-line of verifiers for software product lines. In: SPLC 2013 Workshops, pp. 141–146. ACM (2013)Google Scholar
  11. 11.
    Devroey, X., Cordy, M., Perrouin, G., Kang, E.-Y., Schobbens, P.-Y., Heymans, P., Legay, A., Baudry, B.: A vision for behavioural model-driven validation of software product lines. In: Margaria, T., Steffen, B. (eds.) ISoLA 2012, Part I. LNCS, vol. 7609, pp. 208–222. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Devroey, X., Perrouin, G., Cordy, M., Schobbens, P.Y., Legay, A., Heymans, P.: Towards statistical prioritization for software product lines testing. In: VaMoS 2014, pp. 10:1–10:7. ACM (2013)Google Scholar
  13. 13.
    Fantechi, A., Gnesi, S.: Formal modeling for product families engineering. In: SPLC 2008, pp. 193–202 (2008)Google Scholar
  14. 14.
    Flores, R., Krueger, C., Clements, P.: Mega-scale product line engineering at general motors. In: SPLC 2012, pp. 259–268. ACM (2012)Google Scholar
  15. 15.
    Gruler, A., Leucker, M., Scheidemann, K.: Modeling and model checking software product lines. In: Barthe, G., de Boer, F.S. (eds.) FMOODS 2008. LNCS, vol. 5051, pp. 113–131. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  16. 16.
    Henard, C., Papadakis, M., Perrouin, G., Klein, J., Traon, Y.L.: Multi-objective test generation for software product lines. In: SPLC 2013, pp. 62–71. ACM (2013)Google Scholar
  17. 17.
    Johansen, M.F., Haugen, Ø., Fleurey, F., Eldegard, A.G., Syversen, T.: Generating better partial covering arrays by modeling weights on sub-product lines. In: France, R.B., Kazmeier, J., Breu, R., Atkinson, C. (eds.) MODELS 2012. LNCS, vol. 7590, pp. 269–284. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Kang, K.C., Cohen, S.G., Hess, J.A., Novak, W.E., Spencer Peterson, A.: Feature-Oriented domain analysis (FODA) feasibility study. Tech. rep., Soft. Eng. Inst., Carnegie Mellon Univ. (1990)Google Scholar
  19. 19.
    Kästner, C., von Rhein, A., Erdweg, S., Pusch, J., Apel, S., Rendel, T., Ostermann, K.: Toward variability-aware testing. In: FOSD 2012, pp. 1–8. ACM (2012)Google Scholar
  20. 20.
    Kim, C.H.P., Khurshid, S., Batory, D.S.: Shared execution for efficiently testing product lines. In: ISSRE 2012, pp. 221–230 (2012)Google Scholar
  21. 21.
    Lochau, M., Schaefer, I., Kamischke, J., Lity, S.: Incremental model-based testing of delta-oriented software product lines. In: Brucker, A.D., Julliand, J. (eds.) TAP 2012. LNCS, vol. 7305, pp. 67–82. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  22. 22.
    Mathur, A.P.: Foundations of software testing. Pearson Education (2008)Google Scholar
  23. 23.
    Nguyen, H.V., Kästner, C., Nguyen, T.N.: Exploring variability-aware execution for testing plugin-based web applications. In: ICSE 2014. IEEE (2014)Google Scholar
  24. 24.
    Oster, S., Markert, F., Ritter, P.: Automated incremental pairwise testing of software product lines. In: Bosch, J., Lee, J. (eds.) SPLC 2010. LNCS, vol. 6287, pp. 196–210. Springer, Heidelberg (2010)Google Scholar
  25. 25.
    Perrouin, G., Oster, S., Sen, S., Klein, J., Baudry, B., Traon, Y.L.: Pairwise testing for software product lines: comparison of two approaches. Software Quality Journal 20(3-4), 605–643 (2012)CrossRefGoogle Scholar
  26. 26.
    von Rhein, A., Apel, S., Kästner, C., Thüm, T., Schaefer, I.: The pla model: On the combination of product-line analyses. In: VaMoS 2013, p. 14. ACM (2013)Google Scholar
  27. 27.
    Samih, H.: Relating Variability Modeling and Model-Based Testing for Software Product Lines Testing. In: Weise, C., Nielsen, B. (eds.) Proceedings of the ICTSS 2012 Ph.D. Workshop, pp. 18–22. Aalborg University, Department of Computer Science, Aalborg, Denmark (2012)Google Scholar
  28. 28.
    Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) FORTEST. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  29. 29.
    Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann (2007)Google Scholar
  30. 30.
    Uzuncaova, E., Khurshid, S., Batory, D.: Incremental test generation for software product lines. IEEE Transactions on Software Engineering 36(3), 309–322 (2010)CrossRefGoogle Scholar
  31. 31.
    Whittaker, J.A., Thomason, M.G.: A markov chain model for statistical software testing. TSE 20(10), 812–824 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Xavier Devroey
    • 1
  • Gilles Perrouin
    • 1
  • Axel Legay
    • 2
  • Maxime Cordy
    • 1
  • Pierre-Yves Schobbens
    • 1
  • Patrick Heymans
    • 1
  1. 1.PReCISE Research Center, Faculty of Computer ScienceUniversity of NamurBelgium
  2. 2.INRIA Rennes Bretagne AtlantiqueFrance

Personalised recommendations