Practical Adoptions of T-Way Strategies for Interaction Testing

  • Kamal Z. Zamli
  • Rozmie R. Othman
  • Mohammed I. Younis
  • Mohd Hazli Mohamed Zabil
Part of the Communications in Computer and Information Science book series (CCIS, volume 181)

Abstract

This paper discusses the practical adoption of t-way strategies (also termed interaction testing) for interaction testing. Unlike earlier work, this paper also highlights and unifies the different possible use of t-way strategies including uniform interaction, variable strength interaction, and input-output based relations. In order to help engineers make informed decision on the different use of t-way strategies, this paper discusses the main issues and shortcomings to be considered as well as demonstrates some practical results with a-step-by-step example. In doing so, this paper also analyzes the related works highlighting the current state-of-the-arts and capabilities of some of the existing t-way strategy implementations.

Keywords

software testing interaction testing t-way strategies multi-way testing combinatorial testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Zamli, K.Z., Younis, M.I., Abdullah, S.A.C., Soh, Z.H.C.: Software Testing, 1st edn. Open University, Malaysia KL (2008)Google Scholar
  2. 2.
    Kuhn, D.R., Lei, Y., Kacker, R.: Practical Combinatorial Testing: Beyond Pairwise. IEEE IT Professional 10(3), 19–23 (2008)CrossRefGoogle Scholar
  3. 3.
    Cohen, M. B.: Designing Test Suites For Software Interaction Testing. PhD Thesis, School of Computer Science, University of Auckland (2004)Google Scholar
  4. 4.
    Zekaoui, L.: Mixed Covering Arrays On Graphs And Tabu Search Algorithms. Msc Thesis, Ottawa-Carleton Institute for Computer Science, University of Ottawa, Ottawa, Canada (2006)Google Scholar
  5. 5.
    Shiba, T., Tsuchiya, T., Kikuno, T.: Using Artificial Life Techniques To Generate Test Cases For Combinatorial Testing. In: Proceedings of the 28th Annual Intl. Computer Software and Applications Conf. (COMPSAC 2004), Hong Kong, pp. 72-77 (2004)Google Scholar
  6. 6.
    Younis, M.I., Zamli, K.Z., Klaib, M.F.J., Soh, Z.H.C., Abdullah, S.A.C., Isa, N.A.M.: Assessing IRPS As An Efficient Pairwise Test Data Generation Strategy. International Journal of Advanced Intelligence Paradigms 2(3), 90–104 (2010)CrossRefGoogle Scholar
  7. 7.
    Krishnan, R., Krishna, S.M., Nandhan, P.S.: Combinatorial Testing: Learnings From Our Experience. ACM SIGSOFT Software Engineering Notes 32(3), 1–8 (2007)CrossRefGoogle Scholar
  8. 8.
    Younis, M.I., Zamli, K.Z., Isa, N.A.M.: IRPS: An Efficient Test Data Generation Strategy For Pairwise Testing. In: Proceedings of the 12th International Conference on Knowledge-Based Intelligent Information and Engineering Systems, Part I, pp. 493–500. Springer, Zagreb, Croatia (2008)Google Scholar
  9. 9.
    Allpairs Test Case Generation Tool, http://www.satisfice.com/tools.shtml
  10. 10.
    Lei, Y., Tai, K.C.: In-Parameter-Order: A Test Generation Strategy For Pairwise Testing. In: Proceedings of 3rd IEEE International Conference on High Assurance Systems Engineering Symposium, Washington DC, USA, pp.254–261 (1998)Google Scholar
  11. 11.
    Tung, Y.W., Aldiwan, W.S.: Automatic Test Case Generation For The New Generation Mission Software System. In: Proceedings of IEEE Aerospace Conference, pp. 431–437. Big Sky, MT, USA (March 2000)Google Scholar
  12. 12.
    Harrell, J.M.: Orthogonal Array Testing Strategy (OATS) Technique: Seilevel, Inc. (2001)Google Scholar
  13. 13.
    Daich, G.T.: Testing Combinations Of Parameters Made Easy (Software Testing). In: Proceedings of IEEE Systems Readiness Technology Conference (AUTOTESTCON 2003), pp. 379–384 (2003)Google Scholar
  14. 14.
    Colbourn, C.J., Cohen, M.B., Turban, R.C.: A Deterministic Density Algorithm For Pairwise Interaction Coverage. In: Proceedings. of the Intl. Conference on Software Engineering (IASTED 2004), pp. 345–352 (2004)Google Scholar
  15. 15.
    Lehmann, E., Wegener, J.: Test Case Design By Means Of The CTE-XL. In: Proceedings of the 8th European International Conference on Software Testing, Analysis & Review (EuroSTAR 2000), Copenhagen, Denmark (2000)Google Scholar
  16. 16.
    Copeland, L.: A Practitioner’s Guide To Software Test Design. Massachusetts, STQE Publishing, USA (2004)Google Scholar
  17. 17.
  18. 18.
    Kuhn, D.R., Wallace, D.R., Gallo, A.M.: Software Fault Interaction And Implication For Software Testing. IEEE Transaction on Software Engineering. 30(6), 418–421 (2004)CrossRefGoogle Scholar
  19. 19.
    Younis, M.I., Zamli, K.Z.: Assessing Combinatorial Interaction Strategy For Reverse Engineering Of Combinational Circuits. In: Proceedings of the IEEE Symposium on Industrial Electronics and Applications (ISIEA 2009), Kuala Lumpur, Malaysia (2009)Google Scholar
  20. 20.
    Younis, M.I., Zamli, K.Z.: A Strategy for Automatic Quality Signing And Verification Processes For Hardware And Software Testing. Advances in Software Engineering 1–7 (2010)Google Scholar
  21. 21.
    Zamli, K.Z., Klaib, M.F.J., Younis, M.I., Isa, N.A.M., Abdullah, R.: Design And Implementation Of A T-Way Test Data Generation Strategy With Automated Execution Tool Support. Information Sciences 181(9), 1741–1758 (2011)CrossRefGoogle Scholar
  22. 22.
    Klaib, M. F. J.: Development Of An Automated Test Data Generation And Execution Strategy Using Combinatorial Approach. PhD. Thesis, School of Electrical And Electronics, Universiti Sains Malaysia (2009)Google Scholar
  23. 23.
    IBM Intelligent Test Case Handler, http://www.alphaworks.ibm.com/tech/whitch
  24. 24.
  25. 25.
    Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG System: An Approach To Testing Based On Combinatorial Design. IEEE Transactions on Software Engineering 23(7), 437–444 (1997)CrossRefGoogle Scholar
  26. 26.
    Lei, Y., Kacker, R., Kuhn, D.R., Okun, V., Lawrence, J.: IPOG: A General Strategy For T-Way Software Testing. In: Proceedings of the 14th Annual IEEE International Conference and Workshops on The Engineering of Computer-Based Systems, Tucson, AZ, pp. 549–556 (2007)Google Scholar
  27. 27.
    Lei, Y., Kacker, R., Kuhn, R., Okun, V., Lawrence, J.: IPOG/IPOG-D: Efficient Test Generation For Multi-way Combinatorial Testing. Journal of Software Testing, Verification and Reliability 18(3), 125–148 (2008)CrossRefGoogle Scholar
  28. 28.
    Forbes, M., Lawrence, J., Lei, Y., Kacker, R., Kuhn, D.R.: Refining The In-Paramater-Order Strategy For Constructing Covering Arrays. Journal of Research of the National Institute of Standards and Technology. 113(5), 287–297 (2008)CrossRefGoogle Scholar
  29. 29.
    Younis, M.I., Zamli, K.Z., Isa, N.A.M.: MIPOG - Modification Of The IPOG Strategy For T-Way Software Testing. In: Proceeding of The Distributed Frameworks and Applications (DFmA), Penang, Malaysia (2008)Google Scholar
  30. 30.
    Younis, M. I.: MIPOG: A Parallel T-Way Minimization Strategy For Combinatorial Testing. PhD. Thesis, School of Electrical And Electronics, Universiti Sains Malaysia (2010)Google Scholar
  31. 31.
    Younis, M.I., Zamli, K.Z.: MC-MIPOG: A Parallel T-Way Test Generation Strategy For Multicore Systems. ETRI Journal 32(1), 73–83 (2010)CrossRefGoogle Scholar
  32. 32.
    Younis, M.I., Zamli, K.Z., Isa, N.A.M.: A Strategy For Grid Based T-Way Test Data Generation. In: Proceedings the 1st IEEE International Conference on Distributed Frameworks and Application (DFmA 2008), Penang, Malaysia, pp. 73–78 (2008)Google Scholar
  33. 33.
  34. 34.
    Bryce, R.C., Colbourn, C.J.: A Density-Based Greedy Algorithm For Higher Strength Covering Arrays. Software Testing, Verification and Reliability 19(1), 37–53 (2009)CrossRefGoogle Scholar
  35. 35.
    Bryce, R.C., Colbourn, C.J.: The Density Algorithm For Pairwise Interaction Testing. Software Testing, Verification and Reliability. 17(3), 159–182 (2007)CrossRefGoogle Scholar
  36. 36.
    Nie, C., Xu, B., Shi, L., Dong, G.: Automatic Test Generation For N-Way Combinatorial Testing. In: Reussner, R., Mayer, J., Stafford, J.A., Overhage, S., Becker, S., Schroeder, P.J. (eds.) QoSA 2005 and SOQUA 2005. LNCS, vol. 3712, pp. 203–211. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  37. 37.
    Cohen, M.B., Gibbons, P.B., Mugridge, W.B., Colbourn, C.J., Collofello, J.S.: Variable Strength Interaction Testing Of Components. In: Proceedings of 27th Annual International Computer Software and Applications Conference, Dallas, USA pp. 413–418 (2003)Google Scholar
  38. 38.
    Wang, Z., Xu, B., Nie, C.: Greedy Heuristic Algorithms To Generate Variable Strength Combinatorial Test Suite. In: Proceedings of the 8th International Conference on Quality Software, Oxford, UK, pp. 155–160 (2008)Google Scholar
  39. 39.
    Chen, X., Gu, Q., Li, A., Chen, D.: Variable Strength Interaction Testing With An Ant Colony System Approach. In: Proceedings of 16th Asia-Pacific Software Engineering Conference, Penang, Malaysia, pp. 160–167 (2009)Google Scholar
  40. 40.
    Schroeder, P.J., Korel, B.: Black-Box Test Reduction Using Input-Output Analysis. SIGSOFT Software Engineering Notes 25(5), 173–177 (2000)CrossRefGoogle Scholar
  41. 41.
    Schroeder, P. J.: Black-Box Test Reduction Using Input-Output Analysis. PhD Thesis, Department of Computer Science, Illinois Institute of Technology, Chicago, IL,USA (2001)Google Scholar
  42. 42.
    Schroeder, P.J., Faherty, P., Korel, B.: Generating Expected Results For Automated Black-Box Testing. In: Proceedings of 17th IEEE International Conference on Automated Software Engineering (ASE 2002), Edinburgh, Scotland, UK, pp. 139–148 (2002)Google Scholar
  43. 43.
    Wang, Z., Nie, C., Xu, B.: Generating Combinatorial Test Suite For Interaction Relationship. In: Proceedings of 4th International Workshop on Software Quality Assurance (SOQUA 2007), Dubrovnik, Croatia, pp. 55–61 (2007)Google Scholar
  44. 44.
    Czerwonka, J.: Pairwise Testing In Real World. In: Proceedings of 24th Pacific Northwest Software Quality Conference, Portland, Oregon, USA, pp. 419–430 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Kamal Z. Zamli
    • 1
  • Rozmie R. Othman
    • 2
  • Mohammed I. Younis
    • 1
  • Mohd Hazli Mohamed Zabil
    • 2
  1. 1.School of Electrical EngineeringUniversiti Sains Malaysia, Engineering CampusNibong TebalMalaysia
  2. 2.School of Computer and CommunicationUniversiti Malaysia Perlis (UniMAP)KangarMalaysia

Personalised recommendations