Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Future Internet Testing

FITTEST 2013: Future Internet Testing pp 48–58Cite as

Logging to Facilitate Combinatorial System Testing

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 8432))

Abstract

Testing a web application is typically very complicated. Imposing simple coverage criteria such as function or line coverage is often not sufficient to uncover bugs due to incorrect components integration. Combinatorial testing can enforce a stronger criterion, while still allowing the prioritization of test cases in order to keep the overall effort feasible. Combinatorial testing requires the whole testing domain to be classified and formalized, e.g., in terms of classification trees. At the system testing level, these trees can be quite large. This short paper presents our preliminary work to automatically construct classification trees from loggings of the system, and to subsequently calculate the coverage of our test runs against various combinatorial criteria. We use the tool CTE which allows such criteria to be custom specified. Furthermore, it comes with a graphical interface to simplify the specification of new test sequences.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Memon, A.M., Soffa, M.L., Pollack, M.E.: Coverage criteria for GUI testing. In: Proceedings of the 8th European Software Engineering Conference Held Jointly with 9th ACM SIGSOFT International Symposium on Foundations of Software Engineering, ESEC/FSE-9, pp. 256–267. ACM, New York (2001)

    Google Scholar 

  2. Nie, C., Leung, H.: A survey of combinatorial testing. ACM Comput. Surv. 43, 11:1–11:29 (2011)

    Article  Google Scholar 

  3. Grochtmann, M., Grimm, K.: Classification trees for partition testing. Softw. Test. Verif. Reliab. 3(2), 63–82 (1993)

    Article  Google Scholar 

  4. Musa, J.D.: Operational profiles in software-reliability engineering. IEEE Softw. 10(2), 14–32 (1993)

    Article  Google Scholar 

  5. Misra, R.B., Saravana Kumar, K.: Software operational profile based test case allocation using fuzzy logic. Int. J. Autom. Comput. 4(4), 388 (2007)

    Article  Google Scholar 

  6. Bach, J.: Exploratory testing explained (2003). http://www.satisfice.com/articles/et-article.pdf

  7. Cohen, M.B., Dwyer, M.B., Shi, J.: Interaction testing of highly-configurable systems in the presence of constraints. In: ISSTA ’07: Proceedings of the 2007 International Symposium on Software Testing and Analysis, New York, NY, USA, pp. 129–139 (2007)

    Google Scholar 

  8. Lehmann, E., Wegener, J.: Test case design by means of the CTE XL. In: Proceedings of the 8th European International Conference on Software Testing, Analysis and Review (EuroSTAR 2000), Kopenhagen, Denmark, December 2000

    Google Scholar 

  9. Kruse, P.M., Luniak, M.: Automated test case generation using classification trees. Softw. Qual. Prof. 13(1), 4–12 (2010)

    Google Scholar 

  10. Walton, G.H., Poore, J.H., Trammell, C.J.: Statistical testing of software based on a usage model. Softw. Pract. Exper. 25(1), 97–108 (1995)

    Article  Google Scholar 

  11. Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)

    Article  Google Scholar 

  12. Amland, S.: Risk-based testing: risk analysis fundamentals and metrics for software testing including a financial application case study. J. Syst. Softw. 53(3), 287–295 (2000)

    Article  Google Scholar 

  13. Kruse, P.M., Wegener, J.: Test sequence generation from classification trees. In: Proceedings of ICST 2012 Workshops (ICSTW 2012), Montreal, Canada (2012)

    Google Scholar 

  14. Conrad, M., Drr, H., Fey, I., Yap, A.: Model-based generation and structured representation of test scenarios. In: Proceedings of the Workshop on Software-Embedded Systems Testing, Gaithersburg, Maryland, USA (1999)

    Google Scholar 

  15. Prasetya, I.S.W.B., Middelkoop, A., Elyasov, A., Hage, J.: D6.1: FITTEST Logging Approach, Project no. 257574, FITTEST Future Internet Testing (2011)

    Google Scholar 

  16. Tonella, P., Marchetto, A., Nguyen, D.C., Jia, Y., Lakhotia, K., Harman, M.: Finding the optimal balance between over and under approximation of models inferred from execution logs. In: 2012 IEEE 5th International Conference on Software Testing, Verification and Validation, pp. 21–30. IEEE (2012)

    Google Scholar 

  17. Singh, H., Conrad, M., Sadeghipour, S.: Test case design based on Z and the classification-tree method. In: Proceedings of the 1st International Conference on Formal Engineering Methods, ICFEM ’97, pp. 81–90. IEEE Computer Society, Washington, DC (1997)

    Google Scholar 

  18. Hierons, R.M., Harman, M.: Automatically generating information from a Z specification to support the classification tree method. In: Bert, D., Bowen, J.P., King, S., Waldén, M. (eds.) ZB 2003. LNCS, vol. 2651, pp. 388–407. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  19. Feather, M.S.: Rapid application of lightweight formal methods for consistency analyses. IEEE Trans. Softw. Eng. 24(11), 949–959 (1998)

    Article  Google Scholar 

  20. Ducasse, S., Girba, T., Wuyts, R.: Object-oriented legacy system trace-based logic testing. In: 10th European Conference on Software Maintenance and Reengineering (CSMR). IEEE (2006)

    Google Scholar 

  21. Marchetto, A., Tonella, P., Ricca, F.: State-based testing of Ajax web applications. In: ICST, pp. 121–130. IEEE (2008)

    Google Scholar 

  22. Lorenzoli, D., Mariani, L., Pezzè, M.: Automatic generation of software behavioral models. In: 30th International Conference on Software Engineering, pp. 501–510. ACM (2008)

    Google Scholar 

  23. Nguyen, C.D., Marchetto, A., Tonella, P.: Combining model-based and combinatorial testing for effective test case generation. In: Proceedings of International Symposium on Software Testing and Analysis (ISSTA), Minneapolis, Minnesota, USA (2012)

    Google Scholar 

  24. Jain, S., Singh, I., Chandra, A., Zhang, Z.-L., Bronevetsky, G.: Extracting the textual and temporal structure of supercomputing logs. In: Yang, Y., Parashar, M., Muralidhar, R., Prasanna, V.K. (eds.) HiPC, pp. 254–263. IEEE (2009)

    Google Scholar 

  25. Ernst, M.D., Perkins, J.H., Guo, P.J., McCamant, S., Pacheco, C., Tschantz, M.S., Xiao, C.: The Daikon system for dynamic detection of likely invariants. Sci. Comput. Program. 69(1–3), 35–45 (2007)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgment

This work is supported by EU grant ICT-257574 (FITTEST).

Author information

Authors and Affiliations

  1. Berner and Mattner Systemtechnik GmbH, Berlin, Germany

    Peter M. Kruse

  2. Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands

    I. S. Wishnu B. Prasetya, Jurriaan Hage & Alexander Elyasov

Authors
  1. Peter M. Kruse
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. S. Wishnu B. Prasetya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jurriaan Hage
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Elyasov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter M. Kruse .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Valencia, Valencia, Spain

    Tanja E.J. Vos

  2. University College London, London, United Kingdom

    Kiran Lakhotia

  3. Universidad Politécnica de Valencia, Valencia, Spain

    Sebastian Bauersfeld

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Kruse, P.M., Prasetya, I.S.W.B., Hage, J., Elyasov, A. (2014). Logging to Facilitate Combinatorial System Testing. In: Vos, T., Lakhotia, K., Bauersfeld, S. (eds) Future Internet Testing. FITTEST 2013. Lecture Notes in Computer Science(), vol 8432. Springer, Cham. https://doi.org/10.1007/978-3-319-07785-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-07785-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-07784-0

  • Online ISBN: 978-3-319-07785-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation