Faulty Interaction Identification via Constraint Solving and Optimization

  • Jian Zhang
  • Feifei Ma
  • Zhiqiang Zhang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7317)

Abstract

Combinatorial testing (CT) is an important black-box testing method. In CT, the behavior of the system under test (SUT) is affected by several parameters/components. Then CT generates a combinatorial test suite. After the user executes a test suite and starts debugging, some test cases fail and some pass. From the perspective of a black box, the failures are caused by interaction of several parameters. It will be helpful if we can identify a small set of interacting parameters that caused the failures. This paper proposes a new automatic approach to identifying faulty interactions. It uses (pseudo-Boolean) constraint solving and optimization techniques to analyze the execution results of the combinatorial test suite. Experimental results show that the method is quite efficient and it can find faulty combinatorial interactions quickly. They also shed some light on the relation between the size of test suite and the ability of fault localization.

Keywords

Test Suite System Under Test Combinatorial Testing Covering Array Combinatorial Interaction 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: An approach to testing based on combinatorial design. IEEE Trans. on Softw. Eng. 23(7), 437–444 (1997)CrossRefGoogle Scholar
  2. 2.
    Colbourn, C., McClary, D.: Locating and detecting arrays for interaction faults. J. of Combinatorial Optimization 15(1), 17–48 (2011)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Czerwonka, J.: Pairwise testing in realworld: Practical extensions to test case generator. In: Proc. PNSQC 2006, pp. 419–430 (2006)Google Scholar
  4. 4.
    Gebser, M., Kaufmann, B., Neumann, A., Schaub, T.: Conflict-driven answer set solving. In: Proc. IJCAI 2007, pp. 386–392 (2007), http://www.cs.uni-potsdam.de/clasp/
  5. 5.
    Ghandehari, L.S.G., Lei, Y., Xie, T., Kuhn, D.R., Kacker, R.: Identifying failure-inducing combinations in a combinatorial test set. In: Proc. ICST 2012, pp. 370–379 (2012)Google Scholar
  6. 6.
    Kuhn, D.R., Okun, V.: Pseudo-exhaustive testing for software. In: Proc. SEW 2006, pp. 153–158 (2006)Google Scholar
  7. 7.
    Kuhn, D.R., Wallace, D.R., Gallo, A.M.: Software fault interactions and implications for software testing. IEEE Trans. on Softw. Eng. 30(6), 418–421 (2004)CrossRefGoogle Scholar
  8. 8.
    Martínez, C., Moura, L., Panario, D., Stevens, B.: Algorithms to Locate Errors Using Covering Arrays. In: Laber, E.S., Bornstein, C., Nogueira, L.T., Faria, L. (eds.) LATIN 2008. LNCS, vol. 4957, pp. 504–519. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  9. 9.
    Nie, C., Leung, H.: A survey of combinatorial testing. ACM Computing Surveys 43(2) (2011)Google Scholar
  10. 10.
    Shi, L., Nie, C., Xu, B.: A Software Debugging Method Based on Pairwise Testing. In: Sunderam, V.S., van Albada, G.D., Sloot, P.M.A., Dongarra, J. (eds.) ICCS 2005, Part III. LNCS, vol. 3516, pp. 1088–1091. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  11. 11.
    Wang, Z., Xu, B., Chen, L., Xu, L.: Adaptive interaction fault location based on combinatorial testing. In: Proc. QSIC 2010, pp. 495–502 (2010)Google Scholar
  12. 12.
    Yilmaz, C., Cohen, M.B., Porter, A.A.: Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans. on Softw. Eng. 32(1), 20–34 (2006)CrossRefGoogle Scholar
  13. 13.
    Zeller, A., Hildebrandt, R.: Simplifying and isolating failure-inducing input. IEEE Trans. on Soft. Eng., 183–200 (2002)Google Scholar
  14. 14.
    Zhang, Z., Zhang, J.: Characterizing failure-causing parameter interactions by adaptive testing. In: Proc. ISSTA 2011, pp. 331–341 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jian Zhang
    • 1
  • Feifei Ma
    • 1
  • Zhiqiang Zhang
    • 1
  1. 1.State Key Laboratory of Computer Science, Institute of SoftwareChinese Academy of SciencesChina

Personalised recommendations