Skip to main content
SpringerLink
Log in

Heuristically Enhanced IPO Algorithms for Covering Array Generation

  • Conference paper
  • First Online:
Combinatorial Algorithms (IWOCA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12757))

Included in the following conference series:

Abstract

The construction of covering arrays (CAs) with a small number of rows is a difficult optimization problem. CAs generated by greedy methods are often far from optimal, while many metaheuristics and search techniques become inefficient once larger instances are concerned. In this work, we propose to incorporate improvement heuristics directly into the constructing process of widely used in-parameter-order (IPO) algorithms for CA generation. We discuss how this approach can significantly reduce the search space of the heuristics and implement some of the discussed concepts in the SIPO algorithm, which enhances greedy IPO algorithms with Simulated Annealing. Using SIPO, we improved the best known upper bound on the number of rows of binary CAs of strength 6 for 43 different instances.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://matris.sba-research.org/data/sipo.

References

  1. Ahmed, B.S., Zamli, K.Z., Lim, C.P.: Application of particle swarm optimization to uniform and variable strength covering array construction. Appl. Soft Comput. 12(4), 1330–1347 (2012)

    Article  Google Scholar 

  2. Bryce, R.C., Colbourn, C.J.: A density-based greedy algorithm for higher strength covering arrays. Softw. Test. Verif. Reliab. 19(1), 37–53 (2009)

    Article  Google Scholar 

  3. Bush, K.A., et al.: Orthogonal arrays of index unity. Ann. Math. Stat. 23(3), 426–434 (1952)

    Article  MathSciNet  Google Scholar 

  4. Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: an approach to testing based on combinatorial design. IEEE Trans. Softw. Eng. 23(7), 437–444 (1997)

    Article  Google Scholar 

  5. Colbourn, C.J.: Covering Array Tables for t = 2, 3, 4, 5, 6. http://www.public.asu.edu/~ccolbou/src/tabby/catable.html. Accessed 28 July 2020

  6. Colbourn, C.J., Lanus, E., Sarkar, K.: Asymptotic and constructive methods for covering perfect hash families and covering arrays. Des. Codes Cryptogr. 86(4), 907–937 (2018)

    Article  MathSciNet  Google Scholar 

  7. Duan, F., Lei, Y., Yu, L., Kacker, R.N., Kuhn, D.R.: Improving IPOG’s vertical growth based on a graph coloring scheme. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–8 (2015)

    Google Scholar 

  8. Forbes, M., Lawrence, J., Lei, Y., Kacker, R.N., Kuhn, D.R.: Refining the in-parameter-order strategy for constructing covering arrays. J. Res. Natl. Inst. Stand. Technol. 113(5), 287 (2008)

    Article  Google Scholar 

  9. Hnich, B., Prestwich, S.D., Selensky, E., Smith, B.M.: Constraint models for the covering test problem. Constraints 11(2), 199–219 (2006)

    Article  MathSciNet  Google Scholar 

  10. Jarman, D., et al.: Applying combinatorial testing to large-scale data processing at adobe. In: 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 190–193 (2019)

    Google Scholar 

  11. Kampel, L., Leithner, M., Simos, D.E.: Sliced AETG: a memory-efficient variant of the AETG covering array generation algorithm. Optim. Lett. 14(6), 1543–1556 (2020)

    Article  MathSciNet  Google Scholar 

  12. Kampel, L., Simos, D.E.: A survey on the state of the art of complexity problems for covering arrays. Theor. Comput. Sci. 800, 107–124 (2019)

    Article  MathSciNet  Google Scholar 

  13. Kleine, K., Simos, D.E.: An efficient design and implementation of the in-parameter-order algorithm. Math. Comput. Sci. 12(1), 51–67 (2018)

    Article  MathSciNet  Google Scholar 

  14. Kuhn, D., Kacker, R., Lei, Y.: Introduction to Combinatorial Testing. Chapman & Hall/CRC. Innovations in Software Engineering and Software Development Series. Taylor & Francis (2013)

    Google Scholar 

  15. Lei, Y., Kacker, R., Kuhn, D.R., Okun, V., Lawrence, J.: IPOG: a general strategy for t-way software testing. In: 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS 2007), pp. 549–556 (March 2007)

    Google Scholar 

  16. Lei, Y., Tai, K.C.: In-parameter-order: a test generation strategy for pairwise testing. In: Proceedings Third IEEE International High-Assurance Systems Engineering Symposium (Cat. No. 98EX231), pp. 254–261 (November 1998)

    Google Scholar 

  17. Leithner, M., Kleine, K., Simos, D.E.: CAmetrics: a tool for advanced combinatorial analysis and measurement of test sets. In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 318–327 (2018)

    Google Scholar 

  18. Forbes, M., Lawrence, J., Lei, Y., Kacker, R., Kuhn, D.: NIST repository of CAs. https://math.nist.gov/coveringarrays/ipof/ipof-results.html

  19. Petke, J., Cohen, M.B., Harman, M., Yoo, S.: Practical combinatorial interaction testing: empirical findings on efficiency and early fault detection. IEEE Trans. Softw. Eng. 41(9), 901–924 (2015)

    Article  Google Scholar 

  20. Shiba, T., Tsuchiya, T., Kikuno, T.: Using artificial life techniques to generate test cases for combinatorial testing. In: Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004, COMPSAC 2004, vol. 1, pp. 72–77 (2004)

    Google Scholar 

  21. Torres-Jimenez, J., Izquierdo-Marquez, I., Avila-George, H.: Methods to construct uniform covering arrays. IEEE Access 7, 42774–42797 (2019)

    Article  Google Scholar 

  22. Torres-Jimenez, J., Avila-George, H.: Search-based software engineering to construct binary test-suites. In: Mejia, J., Muñoz, M., Rocha, Á., Calvo-Manzano, J. (eds.) Trends and Applications in Software Engineering. AISC, vol. 405, pp. 201–212. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-26285-7_17

    Chapter  Google Scholar 

  23. Torres-Jimenez, J., Rodriguez-Cristerna, A.: Metaheuristic post-optimization of the NIST repository of covering arrays. CAAI Trans. Intell. Technol. 2(1), 31–38 (2017)

    Article  Google Scholar 

  24. Torres-Jimenez, J., Rodriguez-Tello, E.: New bounds for binary covering arrays using simulated annealing. Inf. Sci. 185(1), 137–152 (2012)

    Article  Google Scholar 

  25. Wagner, M., Kleine, K., Simos, D.E., Kuhn, R., Kacker, R.: CAgen: a fast combinatorial test generation tool with support for constraints and higher-index arrays. In: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 191–200 (2020)

    Google Scholar 

  26. Wagner, M., Kampel, L., Simos, D.E.: IPO-Q: a quantum-inspired approach to the IPO strategy used in CA generation. In: Slamanig, D., Tsigaridas, E., Zafeirakopoulos, Z. (eds.) MACIS 2019. LNCS, vol. 11989, pp. 313–323. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43120-4_24

    Chapter  Google Scholar 

Download references

Acknowledgement

This research was carried out partly in the context of the Austrian COMET K1 program and publicly funded by the Austrian Research Promotion Agency (FFG) and the Vienna Business Agency (WAW).

Author information

Authors and Affiliations

  1. SBA Research, 1040, Vienna, Austria

    Michael Wagner, Ludwig Kampel & Dimitris E. Simos

Authors
  1. Michael Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ludwig Kampel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dimitris E. Simos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Wagner .

Editor information

Editors and Affiliations

  1. University of Ottawa, Ottawa, ON, Canada

    Paola Flocchini

  2. University of Ottawa, Ottawa, ON, Canada

    Lucia Moura

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wagner, M., Kampel, L., Simos, D.E. (2021). Heuristically Enhanced IPO Algorithms for Covering Array Generation. In: Flocchini, P., Moura, L. (eds) Combinatorial Algorithms. IWOCA 2021. Lecture Notes in Computer Science(), vol 12757. Springer, Cham. https://doi.org/10.1007/978-3-030-79987-8_40

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-79987-8_40

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-79986-1

  • Online ISBN: 978-3-030-79987-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation