On the Application of SAT Solvers to the Test Suite Minimization Problem

  • Franco Arito
  • Francisco Chicano
  • Enrique Alba
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7515)


The Test Suite Minimization problem in regression testing is a software engineering problem which consists in selecting a set of test cases from a large test suite that satisfies a given condition, like maximizing the coverage and/or minimizing the oracle cost. In this work we use an approach based on SAT solvers to find optimal solutions for the Test Suite Minimization Problem. The approach comprises two translations: from the original problem instance into Pseudo-Boolean constraints and then to a propositional Boolean formula. In order to solve a problem, we first translate it into a SAT instance. Then the SAT instance is solved using a state-of-the-art SAT solver. Our main contributions are: we create an encoding for single and multi-objective formulations of the Test Suite Minimization Problem as Pseudo-Boolean constraints and we compute optimal solutions for well-known and highly-used instances of this problem for future reference.


Test suite minimization satisfiability problem multi-objective optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Biere, A., Heule, M., van Maaren, H., Walsh, T.: Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)zbMATHGoogle Scholar
  2. 2.
    Cook, S.A.: The complexity of theorem-proving procedures. In: Proceedings of the Third Annual ACM Symposium on Theory of Computing, STOC 1971, pp. 151–158. ACM, New York (1971)CrossRefGoogle Scholar
  3. 3.
    Davis, M., Logemann, G., Loveland, D.: A machine program for theorem-proving. Commun. ACM 5(7), 394–397 (1962)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Deb, K.: Multi-objective optimization using evolutionary algorithms. John Wiley & Sons (2001)Google Scholar
  5. 5.
    Do, H., Elbaum, S., Rothermel, G.: Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering 10, 405–435 (2005)CrossRefGoogle Scholar
  6. 6.
    Eén, N., Sörensson, N.: An Extensible SAT-solver. In: Giunchiglia, E., Tacchella, A. (eds.) SAT 2003. LNCS, vol. 2919, pp. 502–518. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Eén, N., Sörensson, N.: Translating Pseudo-Boolean Constraints into SAT. Journal on Satisfiability, Boolean Modeling and Computation 2(1-4), 1–26 (2006)zbMATHGoogle Scholar
  8. 8.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1979)zbMATHGoogle Scholar
  9. 9.
    Gomes, C., Selman, B., Crato, N.: Heavy-tailed Distributions in Combinatorial Search. In: Smolka, G. (ed.) CP 1997. LNCS, vol. 1330, pp. 121–135. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  10. 10.
    Hsu, H.Y., Orso, A.: MINTS: A general framework and tool for supporting test-suite minimization. In: IEEE 31st International Conference on Software Engineering, ICSE 2009, pp. 419–429 (May 2009)Google Scholar
  11. 11.
    Hutchins, M., Foster, H., Goradia, T., Ostrand, T.: Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria. In: Proceedings of the 16th International Conference on Software Engineering, ICSE 1994, pp. 191–200. IEEE Computer Society Press, Los Alamitos (1994)CrossRefGoogle Scholar
  12. 12.
    Marques-Silva, J.: Practical applications of boolean satisfiability. In: 9th International Workshop on Discrete Event Systems, WODES 2008, pp. 74 –80 (May 2008)Google Scholar
  13. 13.
    Mitchell, D., Selman, B., Levesque, H.: Hard and easy distributions of SAT problems. In: Proceedings of the Tenth National Conference on Artificial Intelligence, AAAI 1992, pp. 459–465. AAAI Press (1992)Google Scholar
  14. 14.
    Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S.: Chaff: engineering an efficient SAT solver. In: Proceedings of the 38th Annual Design Automation Conference, DAC 2001, pp. 530–535. ACM, New York (2001)CrossRefGoogle Scholar
  15. 15.
    Silva, J.P.M., Sakallah, K.A.: Grasp: A new search algorithm for satisfiability. In: Proceedings of the 1996 IEEE/ACM International Conference on Computer-aided Design, pp. 220–227. IEEE Computer Society, Washington, DC (1996)Google Scholar
  16. 16.
    Stallman, R.M., Sussman, G.J.: Forward reasoning and dependency-directed backtracking in a system for computer-aided circuit analysis. Artificial Intelligence 9(2), 135–196 (1977)zbMATHCrossRefGoogle Scholar
  17. 17.
    Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Software Testing, Verification and Reliability (2010),, doi: 10.1002/stvr.430
  18. 18.
    Zhang, L., Hou, S.S., Guo, C., Xie, T., Mei, H.: Time-aware test-case prioritization using integer linear programming. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, ISSTA 2009, pp. 213–224. ACM, New York (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Franco Arito
    • 1
  • Francisco Chicano
    • 1
  • Enrique Alba
    • 1
  1. 1.Departamento de Lenguajes y Ciencias de la ComputaciónUniversity of MálagaSpain

Personalised recommendations