Skip to main content
SpringerLink
Log in

Supercomputing and grid computing on the verification of covering arrays

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

An Erratum to this article was published on 19 February 2015

Abstract

The Covering Arrays (CAs) are mathematical objects with minimal coverage and maximum cardinality that are a good tool for the design of experiments. A covering array is an N×k matrix over an alphabet v s.t. each N×k subset contains at least one time each combination from {0,1,…,v−1}t, given a positive integer value t. The process of ensuring that a CA contains each of the v t combinations is called verification of CA. In this paper, we present an algorithm for CA verification and its implementation details in three different computation paradigms: (a) sequential approach (SA); (b) parallel approach (PA); and (c) Grid approach (GA). Four different PAs were compared in their performance of verifying a matrix as a CA; the PA with the best performance was included in a different experimentation where the three paradigms, SA, PA, and GA were compared in a benchmark composed by 45 possible CA instances. The results showed the limitations of the different paradigms when solving the verification of CA problem, and points out the necessity of a Grid approach to solve the problem when the size of a CA grows.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 2
Algorithm 7
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Tirant supercomputer: http://www.uv.es/siuv/cas/zcalculo/res/descripcion.wiki.

  2. European Grid Infrastructure: http://www.egi.eu/.

References

  1. Avila-George H, Torres-Jimenez J, Hernández V, Rangel-Valdez N (2010) Verification of general and cyclic covering arrays using Grid computing. In: Proceedings of the 3rd international conference on data management in grid and peer-to-peer systems, GLOBE 2010, Bilbao, Spain, 30 August–3 September. Lecture notes in computer science, vol 6265. Springer, Berlin, pp 112–123. doi:10.1007/978-3-642-15108-8_10

    Google Scholar 

  2. Bryce RC, Colbourn CJ (2007) The density algorithm for pairwise interaction testing. Softw Test Verif Reliab 17(3):159–182. doi:10.1002/stvr.365

    Article  Google Scholar 

  3. Burr K, Young W (1998) Combinatorial test techniques: table-based automation, test generation and code coverage. In: Proceedings of the international conference on software testing, analysis, and review—STAR, pp 503–513. West, 1998

    Google Scholar 

  4. Bush KA (1952) Orthogonal arrays of index unity. Ann Math Stat 23(3):426–434. doi:10.1214/aoms/1177729387

    Article  MathSciNet  MATH  Google Scholar 

  5. Calvagna A, Gargantini A, Tramontana E (2009) Building T-wise combinatorial interaction test suites by means of grid computing. In: Proceedings of the 18th IEEE international workshops on enabling technologies: infrastructures for collaborative enterprises—WETICE 2009. IEEE Comput Soc, Los Alamitos, pp 213–218. doi:10.1109/WETICE.2009.52

    Chapter  Google Scholar 

  6. Cawse JN (2003) Experimental design for combinatorial and high throughput materials development. Wiley, New York

    Google Scholar 

  7. Cheng C (2007) The test suite generation problem: optimal instances and their implications. Discrete Appl Math 155:1943–1957. doi:10.1016/j.dam.2007.04.010

    Article  MathSciNet  MATH  Google Scholar 

  8. Cohen DM, Dalal SR, Parelius J, Patton GC (1996) The combinatorial design approach to automatic test generation. IEEE Softw 13(5):83–88. doi:10.1109/52.536462

    Article  Google Scholar 

  9. Cohen MB, Colbourn CJ, Ling ACH (2003) Augmenting simulated annealing to build interaction test suites. In: Proceedings of the 14th international symposium on software reliability engineering—ISSRE 2003. IEEE Comput Soc, Los Alamitos, pp 394–405. http://doi.ieeecomputersociety.org/10.1109/ISSRE.2003.1251061

    Chapter  Google Scholar 

  10. Colbourn CJ (2004) Combinatorial aspects of covering arrays. Matematiche 59(1, 2):125–172

    MathSciNet  MATH  Google Scholar 

  11. Colbourn CJ (2011) Covering array tables for t=2,3,4,5,6. URL: http://www.public.asu.edu/~ccolbou/src/tabby/catable.html. Accessed on April 20

  12. Colbourn CJ, Kéri G (2009) Binary covering arrays and existentially closed graphs. In: Proceedings of the 2nd international workshop on coding and cryptology—IWCC 2009. Lecture notes in computer science, vol 5557. Springer, Berlin, pp 22–33. doi:10.1007/978-3-642-01877-0_3

    Google Scholar 

  13. Colbourn CJ, Ling ACH (2009) A recursive construction for perfect hash families. J Math Cryptol 3(4):291–306. doi:10.1515/JMC.2009.018

    Article  MathSciNet  MATH  Google Scholar 

  14. Colbourn CJ, Martirosyan SS, Mullen GL, Shasha D, Sherwood GB, Yucas JL (2006) Products of mixed covering arrays of strength two. J Comb Des 12(2):124–138. doi:10.1002/jcd.20065

    Article  MathSciNet  Google Scholar 

  15. Colbourn CJ, Torres-Jimenez J (2010) Heterogeneous hash families and covering arrays. In: Error-correcting codes, finite geometries and cryptography. Contemporary mathematics, vol 523. pp 3–15. ISBN-10 0-8218-4956-5

    Chapter  Google Scholar 

  16. DIANE (2011) Distributed analysis environment. URL: http://it-proj-diane.web.cern.ch/it-proj-diane/. Accessed on June 6

  17. Foster I, Kesselman C (1999) The grid: blueprint for a new computing infrastructure. Morgan Kaufmann, San Mateo

    Google Scholar 

  18. Seroussi NBG (1988) Vector sets for exhaustive testing of logic circuits. IEEE Trans Inf Theory 34:513–522

    Article  MathSciNet  MATH  Google Scholar 

  19. Gonzalez-Hernandez L, Rangel-Valdez N, Torres-Jimenez J (2010) Construction of mixed covering arrays of variable strength using a tabu search approach. In: Proceedings of the 4th international conference on combinatorial optimization and applications, COCOA, 2010. Lecture notes in computer science, vol 6508. Springer, Berlin, pp 51–64. doi:10.1007/978-3-642-17458-2_6

    Chapter  Google Scholar 

  20. Hedayat AS, Sloane NJA, Stufken J (1999) Orthogonal arrays: theory and applications. Springer, Berlin

    MATH  Google Scholar 

  21. Katona GOH (1973) Two applications (for search theory and truth functions) of Sperner type theorems. Period Math Hung 3(1–2):19–26. doi:10.1007/BF02018457

    Article  MathSciNet  MATH  Google Scholar 

  22. Kleitman DJ, Spencer J (1973) Families of k-independent sets. Discrete Math 6(3):255–262. doi:10.1016/0012-365X(73)90098-8

    Article  MathSciNet  MATH  Google Scholar 

  23. Kuhn R, Lei Y, Kacker R (2008) Practical combinatorial testing: beyond pairwise. IT Prof 10(3):19–23. doi:10.1109/MITP.2008.54

    Article  Google Scholar 

  24. Lawrence J, Kacker R, Lei Y, Kuhn D, Forbes M (2011) A survey of binary covering arrays. Electron J Comb 18(1):84

    MathSciNet  Google Scholar 

  25. Lei Y, Kacker R, Kuhn DR, Okun V, Lawrence J (2007) IPOG: a general strategy for t-way software testing. In: Proceedings of the 14th annual IEEE international conference and workshops on the engineering of computer-based systems—ECBS 2007. IEEE Comput Soc, Los Alamitos, pp 549–556. doi:10.1109/ECBS.2007.47

    Google Scholar 

  26. Martinez-Pena J, Torres-Jimenez J, Rangel-Valdez N, Avila-George H (2010) A heuristic approach for constructing ternary covering arrays using trinomial coefficients. In: Proceedings of the 12th Ibero-American conference on artificial intelligence—IBERAMIA 2010. Lecture notes in computer science, vol 6433. Springer, Berlin, pp 572–581. doi:10.1007/978-3-642-16952-6_58

    Chapter  Google Scholar 

  27. Martirosyan SS, Colbourn CJ (2005) Recursive constructions of covering arrays. Bayreuth Math Schr 74:266–275

    MathSciNet  MATH  Google Scholar 

  28. McDowell AG (2011) All-pairs testing. URL: http://www.mcdowella.demon.co.uk/allPairs.html. Accessed on June 21

  29. Moscicki J, Brochu F, Ebke J, Egede U, Elmsheuser J, Harrison K, Jones R, Lee H, Liko D, Maier A, Muraru A, Patrick G, Pajchel K, Reece W, Samset B, Slater M, Soroko A, Tan C, van der Ster D, Williams M (2009) Ganga: a tool for computational-task management and easy access to grid resources. Comput Phys Commun 180(11):2303–2316. doi:10.1016/j.cpc.2009.06.016

    Article  Google Scholar 

  30. Moura L, Stardom J, Stevens B, Williams A (2003) Covering arrays with mixed alphabet sizes. J Comb Des 11(6):413–432. doi:10.1002/jcd.10059

    MathSciNet  MATH  Google Scholar 

  31. National Institute of Standards and Technology (2011) NIST covering array tables. URL: http://math.nist.gov/coveringarrays/. Accessed on April 20

  32. Nurmela KJ (2004) Upper bounds for covering arrays by tabu search. Discrete Appl Math 138:143–152. doi:10.1016/S0166-218X(03)00291-9

    Article  MathSciNet  MATH  Google Scholar 

  33. Pacini F (2011) Job description language howto. URL: http://server11.infn.it/workload-grid/docs/DataGrid-01-TEN-0102-0_2-Document.pdf. Accessed on October 10

  34. Phadke MS (1995) Quality engineering using robust design. Prentice Hall, New York

    Google Scholar 

  35. Rényi A (1971) Foundations of probability. Wiley, New York

    Google Scholar 

  36. Shasha DE, Kouranov AY, Lejay LV, Chou MF, Coruzzi GM (2001) Using combinatorial design to study regulation by multiple input signals: a tool for parsimony in the post-genomics era. Plant Physiol 127(4):1590–1594. doi:10.1104/pp.010683

    Article  Google Scholar 

  37. Sherwood GB (2008) Optimal and near-optimal mixed covering arrays by column expansion. Discrete Math 308(24):6022–6035. doi:10.1016/j.disc.2007.11.021

    Article  MathSciNet  MATH  Google Scholar 

  38. Sherwood GB (2011) On the construction of orthogonal arrays and covering arrays using permutation groups. URL: http://testcover.com/pub/background/cover.htm. Accessed on June 20

  39. Shiba T, Tsuchiya T, Kikuno T (2004) Using artificial life techniques to generate test cases for combinatorial testing. In: Proceedings of the 28th annual international computer software and applications conference, vol 01, COMPSAC 2004. IEEE Comput Soc, Los Alamitos, pp 72–77. doi:10.1109/CMPSAC.2004.1342808

    Chapter  Google Scholar 

  40. Torres-Jimenez J, Avila-George H, Rangel-Valdez N, Gonzalez-Hernandez L (2012) Construction of orthogonal arrays of index unity using logarithm tables for galois fields. In Cryptography. InTech, pp 71–90. ISBN 978-953-51-0179-6

    Google Scholar 

  41. Torres-Jimenez J, De Alfonso C, Hernández V (2004) Computation of ternary covering arrays using a grid. In: Proceedings of the second Asian applied computing conference—AACC 2004. Lecture notes in computer science, vol 3285. Springer, Berlin, pp 240–246. doi:10.1007/978-3-540-30176-9_31

    Google Scholar 

  42. Torres-Jimenez J, Rangel-Valdez N, Gonzalez-Hernandez AL, Avila-George H (2010) Construction of logarithm tables for Galois fields. Int J Math Educ Sci Technol 42(1):91–102. doi:10.1080/0020739X.2010.510215

    Article  MathSciNet  Google Scholar 

  43. Torres-Jimenez J, Rodriguez-Tello E (2012) New bounds for binary covering arrays using simulated annealing. Inf Sci 185(1):137–152. doi:10.1016/j.ins.2011.09.020

    Article  Google Scholar 

  44. Tung Y, Aldiwan WS (2000) Automating test case generation for the new generation mission software system. In: Proceedings of the IEEE aerospace conference, vol 1. IEEE Press, New York, pp 431–437. doi:10.1109/AERO.2000.879426

    Google Scholar 

  45. Vadde K, Syrotiuk V (2004) Factor interaction on service delivery in mobile ad hoc networks. IEEE J Sel Areas Commun 22(7):1335–1346. doi:10.1109/JSAC.2004.829351

    Article  Google Scholar 

  46. Williams AW (2000) Determination of test configurations for pair-wise interaction coverage. In: Proceedings of the IFIP TC6/WG6.1 13th International conference on testing communicating systems: tools and techniques—TestCom. Kluwer, Norwell, pp 59–74

    Google Scholar 

  47. Williams AW, Probert RL (1996) A practical strategy for testing pair-wise coverage of network interfaces. In: Proceedings of the seventh international symposium on software reliability engineering—ISSRE. IEEE Comput Soc, Los Alamitos, pp 246–256. doi:10.1109/ISSRE.1996.558835

    Chapter  Google Scholar 

  48. Yilmaz C, Cohen MB, Porter AA (2006) Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans Softw Eng 32(1):20–34. doi:10.1109/TSE.2006.8

    Article  Google Scholar 

  49. Younis M, Zamli K, Isa N (2008) A strategy for grid based t-way test data generation. In: First international conference on distributed framework and applications—DFmA 2008, pp 73–78. doi:10.1109/ICDFMA.2008.4784416

    Chapter  Google Scholar 

Download references

Acknowledgements

The authors thankfully acknowledge the computer resources and assistance provided by Spanish Supercomputing Network (TIRANT-UV). This research work was partially funded by the following projects: CONACyT 58554, Calculo de Covering Arrays; 51623 Fondo Mixto CONACyT y Gobierno del Estado de Tamaulipas.

Author information

Authors and Affiliations

  1. Instituto de Instrumentación para Imagen Molecular (I3M), Centro mixto CSIC, Universitat Politècnica de València, CIEMAT, Camino de Vera S/N, 46022, Valencia, Spain

    Himer Avila-George, Abel Carrión & Vicente Hernández

  2. Information Technology Laboratory, CINVESTAV-Tamaulipas, Km. 5.5 Carretera Victoria-Soto La Marina, 87130, Victoria Tamps, Mexico

    Jose Torres-Jimenez

  3. Universidad Politécnica de Ciudad Victoria, Parque Científico y Tecnológico de Tamaulipas, Av. Nuevas Tecnologias 5902, Carretera Victoria-Soto la Marina Km. 5.5, 87138, Cd. Victoria, Tamps., Mexico

    Nelson Rangel-Valdez

Authors
  1. Himer Avila-George
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose Torres-Jimenez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nelson Rangel-Valdez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abel Carrión
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vicente Hernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Himer Avila-George.

Additional information

An erratum to this article is available at http://dx.doi.org/10.1007/s11227-015-1389-9.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Avila-George, H., Torres-Jimenez, J., Rangel-Valdez, N. et al. Supercomputing and grid computing on the verification of covering arrays. J Supercomput 62, 916–945 (2012). https://doi.org/10.1007/s11227-012-0763-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-012-0763-0

Keywords

Search

Navigation