Skip to main content
SpringerLink
Log in

Covering Arrays Generation Methods Survey

  • Conference paper
Book cover Leveraging Applications of Formal Methods, Verification, and Validation (ISoLA 2010)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 6416))

Abstract

This paper is a survey of methods for covering arrays generation. Covering arrays are used in test data generation for program interfaces with many parameters. The effectiveness and the range of application of these methods are analysed. The algorithms used in these methods are analysed for their time complexity and memory usage. In this paper combinatorial, recursive, greedy algorithms are observed. Several heuristics for reducing the size and the time for construction of covering arrays are observed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Stevens, B., Mendelsohn, E.: New recursive methods for transversal covers. Journal of Combinatorial Designs 7(3), 185–203 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  2. Sloane, N.: Covering arrays and intersecting codes. Journal of Combinatorial Designs 1(1), 51–63 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  3. Chateauneuf, M., Kreher, D.: On the state of strength-three covering arrays. Journal of Combinatorial Designs 10(4), 217–238 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  4. Martirosyan, S., Van Trung, T.: Tran Van Trung: On t-covering arrays. Designs, Codes and Cryptography 32, 323–339 (2004)

    Article  MATH  Google Scholar 

  5. Hartman, A., Raskin, L.: Problems and algorithms for covering arrays. Discrete Math. 284, 149–156 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cohen, M., Colbourn, C., Ling, A.: Constructing Strength Three Covering Arrays with Augmented Annealing. Discrete Mathematics 308, 2709–2722 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cohen, D., Dalal, S., Fredman, M., Patton, G.: The AETG System: an approach to testing based on combinatorial design. IEEE Transactions on Software Engineering 23(7), 437–444 (1996)

    Article  Google Scholar 

  8. Nurmela, K.: Upper bounds for covering arrays by tabu search. Discrete Applied Math. 138, 143–152 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Colbourn, C., Martirosyan, S., Van Trung, T., Walker II, R.: Roux-type Constructions for Covering Arrays of Strengths Three and Four Designs. Codes and Cryptography 41(1), 33–57 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  10. Stevens, B.: Transversal Covers and Packings. PhD. Thesis, Mathematics, University of Toronto (1998)

    Google Scholar 

  11. Patton, G.: DAT (Defect Analysis Team)1986-1990 Overview. Internal Bellcore Technical Memo (1991)

    Google Scholar 

  12. Cohen, M., Colbourn, C., Ling, A.: Augmenting Sumulated Annealing to Build Interaction Test Suites. In: Proc Intl. Symposium Software Requirements Engineering, pp. 394–405 (2003)

    Google Scholar 

  13. Dueck, G.: New Optimization Heuristic – The Great Deluge Algorithm and the Record-To-Record Travel. Journal of Computational Physics 104, 86–92 (1993)

    Article  MATH  Google Scholar 

  14. Stevens, B., Ling, A., Mendelsohn, E.: A direct construction of transversal coversusing group divisible designs. Ars. Combin. 63, 145–159 (2002)

    MathSciNet  MATH  Google Scholar 

  15. Dueck, G., Scheuer, T.: Threshold Accepting: A general purpose optimization algorithm appearing superior to simulating annealing. Journal of Computational Physics 90, 161–175 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  16. Stardom, J.: Metaheuristic and the search for covering and packing arrays. Master’s thesis, Simon Fraser University (2001)

    Google Scholar 

  17. Jenkins, B.: Tool for pairwise testing (2005), http://burtleburtle.net/bob/math/jenny.html

  18. Lei, Y., Tai, K.: In-parameter order: A test generation strategy for pairwise testing. In: Proc. 3rd IEEE High Assurance System Engineering Symposium, pp. 254–161 (1998)

    Google Scholar 

  19. Seroussi, G., Bshouty, N.: Vector sets for exhaustive testing of logic circuits. IEEE Trans. Information Theory 34, 513–522 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  20. Cohen, D., Dalal, S., Parelius, J., Patton, G.: The Combinatorial Design Approach to Automatic Test Generation. IEEE Software, 83–87 (September 1996)

    Google Scholar 

  21. Pairwise Testing, Combinatorial Test Case Generation, http://www.pairwise.org/tools.asp

  22. Colbourn, C.: Combinatorial aspects of covering arrays. Le Matematiche (Catania) 58, 121–167 (2004)

    MATH  Google Scholar 

  23. Cohen, D., Dalal, S., Fredman, M., Patton, G.: The Automatic Efficient Test Generator (AETG) System. In: Proceedings of 5th International Symposium on Software Reliability Engineering, November 6-9, pp. 303–309 (1994)

    Google Scholar 

  24. Colbourn, C., Cohen, M., Turban, R.: A deterministic density algorithm for pairwise interaction coverage. In: Proc. of the IASTED Intl. Conference on Software Engineering, pp. 242–252 (February 2004)

    Google Scholar 

  25. Williams, A.: Determination of Test Configurations for Pairwise Interaction Coverage. In: Proceedings of the 13th International Conference on Testing Communicating Systems (Test-Com 2000), pp. 59–74 (2000)

    Google Scholar 

  26. Cormen, T., Stein, C., Rivest, R., Leiserson, C.: Greedy algorithms // Introduction to Algorithms, 2nd edn., vol. 16. Moscow Viliyams (2006)

    Google Scholar 

  27. Grindal, M., Offutt, A., Andler, S.: Combination testing strategies: A survey. Software Testing, Verification, and Reliability 15(3), 167–199 (2005)

    Article  Google Scholar 

  28. Hartman, A.: Software and Hardware Testing Using Combinatorial Covering Suites. In: Proc. Graph Theory, Combinatorics and Algorithms: Interdisciplinary Applications, pp. 266–327 (2005)

    Google Scholar 

  29. Edelman, A.: The mathematics of the Pentium division bug. SIAM Review 39, 54–67 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  30. Greene, C.: Sperner families and partitions of a partially ordered set. In: Hall Jr., M., van Lint, J. (eds.) Combinatorics, Dordrecht, Holland, pp. 277–290 (1975)

    Google Scholar 

  31. Williams, A., Probert, R.: A measure for component interaction test coverage. In: Proc. ACS/IEEE Intl. Conf. on Computer Systems and Applications, pp. 301–311 (2001)

    Google Scholar 

  32. Ammann, P., Offutt, J.: Using Formal Methods to Derive Test Frames in Category-Partition Testing Safety, Reliability, Fault Tolerance, Concurrency, and Real Time Security. In: Proc. Ninth Ann. Conf. Computer Assurance (COMPASS 1994), pp. 69–79 (1994)

    Google Scholar 

  33. Zelenov, S., Zelenova, S.: Automated negative and positive test generation for testing a syntax analysis phase. Works of Institute for System Programming RAS, russian (2004)

    Google Scholar 

  34. Documentation on Webmoney interface WM Keeper Light, http://www.wmtransfer.com/eng/about/demo/light/index.shtml

  35. Colbourn, C., Martirosyan, S., Mullen, G., Shasha, D., Yucas, J., Sherwood, G.: Products of Mixed Covering Arrays of Strength Two. J. Combin. Des. 14, 124–138 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  36. Tung, Y.-W., Aldiwan, W.: Automating test case generation for the new generation mission software system. In: Proc. IEEE Aerospace Conf., pp. 431–437 (2000)

    Google Scholar 

  37. Bryce, R., Colbourn, C.: One-Test-at-a-Time Heuristic Search for Interaction Test Suites. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), Search-based Software Engineering track (SBSE), London, England, pp. 1082–1089 (2007)

    Google Scholar 

  38. Lidl, R., Niederreiter, H.: Finite fields, vol. 2. Mir, Moscow (1988)

    MATH  Google Scholar 

  39. Tables for the smallest known covering arrays, http://www.public.asu.edu/~ccolbou/src/tabby/catable.html

  40. Godbole, A., Skipper, D., Sunley, R.: t-Covering arrays: upper bounds and Poisson approximations. Combinatorics, Probability and Computing 5, 105–118 (1996)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for System Programming, Russian Academy of Sciences, 25 Alexander Solzhenitsyn st., Moscow, 109004, Russia

    Victor Kuliamin & Alexander Petukhov

Authors
  1. Victor Kuliamin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Petukhov
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute for Informatics, University of Potsdam, August-Bebel-Str. 89, 14482, Potsdam, Germany

    Tiziana Margaria

  2. Technical University of Dortmund, Otto-Hahn-Str. 14, 44227, Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kuliamin, V., Petukhov, A. (2010). Covering Arrays Generation Methods Survey. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification, and Validation. ISoLA 2010. Lecture Notes in Computer Science, vol 6416. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16561-0_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-16561-0_36

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-16560-3

  • Online ISBN: 978-3-642-16561-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation