Advertisement

Generating Regression Unit Tests Using a Combination of Verification and Capture & Replay

  • Christoph Gladisch
  • Shmuel Tyszberowicz
  • Bernhard Beckert
  • Amiram Yehudai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6143)

Abstract

The combination of software verification and testing techniques is increasingly encouraged due to their complementary strengths. Some verification tools have extensions for test case generation. These tests are strong at detecting software faults during the implementation and verification phase, and to further increase the confidence in the final software product. However, tests generated using verification technology alone may lack some of the benefits obtained by using more traditional testing techniques. One such technique is Capture and Replay, whose strengths are the generation of isolated unit tests and regression test oracles.

Hence, the two groups of techniques have complementary strengths, and therefore are ideal candidates for a tool-chain approach proposed in this paper. The first phase produces, for a given system, unit tests with high coverage. However, when using them to test a unit, its environment is tested as well – resulting in a high cost of testing. To solve this problem, the second phase captures the various executions of the program, which are monitored by the output of the first phase. The output of the second phase is a set of unit tests with high code coverage, which uses mock objects to test the units. Another advantage of this approach is the fact that the generated tests can also be used for regression testing.

Keywords

Test Suite Unit Test Symbolic Execution Test Case Generation Proof Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Beckert, B., Gladisch, C.: White-box testing by combining deduction-based specification extraction and black-box testing. In: Gurevich, Y., Meyer, B. (eds.) TAP 2007. LNCS, vol. 4454, pp. 207–216. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Beckert, B., Hähnle, R., Schmitt, P.H. (eds.): Verification of Object-Oriented Software. The KeY Approach. LNCS (LNAI), vol. 4334. Springer, Heidelberg (2007)Google Scholar
  3. 3.
    Burdy, L., Requet, A., Lanet, J.-L.: Java applet correctness: A developer-oriented approach. In: Int. Symp. Formal Methods Europe, pp. 422–439 (2003)Google Scholar
  4. 4.
    Cok, D.R., Kiniry, J.: ESC/Java2: Uniting ESC/Java and JML. In: Barthe, G., Burdy, L., Huisman, M., Lanet, J.-L., Muntean, T. (eds.) CASSIS 2004. LNCS, vol. 3362, pp. 108–128. Springer, Heidelberg (2005)Google Scholar
  5. 5.
    Deng, X., Robby, Hatcliff, J.: Kiasan: A verification and test-case generation framework for Java based on symbolic execution. In: ISoLA, Paphos, Cyprus, p. 137 (2006)Google Scholar
  6. 6.
    Deng, X., Robby, Hatcliff, J.: Kiasan/KUnit: Automatic test case generation and analysis feedback for open object-oriented systems. In: TAICPART-MUTATION 2007: Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION, Washington, DC, USA, pp. 3–12. IEEE Computer Society Press, Los Alamitos (2007)CrossRefGoogle Scholar
  7. 7.
    du Bousquet, L., Ledru, Y., Maury, O., Oriat, C., Lanet, J.-L.: Case study in JML-based software validation. In: Proceedings, Automated Software Engineering, pp. 294–297 (2004)Google Scholar
  8. 8.
    Elbaum, S.G., Chin, H.N., Dwyer, M.B., Jorde, M.: Carving and replaying differential unit test cases from system test cases. IEEE Trans. Software Eng. 35(1), 29–45 (2009)CrossRefGoogle Scholar
  9. 9.
    Engel, C., Gladisch, C., Klebanov, V., Rümmer, P.: Integrating verification and testing of object-oriented software. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 182–191. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Engel, C., Hähnle, R.: Generating Unit Tests from Formal Proofs. In: Gurevich, Y., Meyer, B. (eds.) TAP 2007. LNCS, vol. 4454, pp. 169–188. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Ernst, M.D.: Static and dynamic analysis: synergy and duality. In: Workshop on Program Analysis For Software Tools and Engineering (PASTE), p. 35 (2004)Google Scholar
  12. 12.
    Fowler, M.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Reading (2000)Google Scholar
  13. 13.
    Gladisch, C.: Verification-based testing for full feasible branch coverage. In: Proceedings, Software Engineering and Formal Methods (SEFM). IEEE Computer Society Press, Los Alamitos (2008)Google Scholar
  14. 14.
    Graves, T.L., Harrold, M.J., Kim, J.-M., Porter, A., Rothermel, G.: An empirical study of regression test selection techniques. TOSEM 10(2), 184–208 (2001)zbMATHCrossRefGoogle Scholar
  15. 15.
    Hamill, P.: Unit test frameworks. O’Reilly, Sebastopol (2004)Google Scholar
  16. 16.
    Harrold, M.J., Jones, J.A., Li, T., Liang, D., Orso, A., Pennings, M., Sinha, S., Spoon, S.A., Gujarathi, A.: Regression test selection for Java software. SIGPLAN Not. 36(11), 312–326 (2001)CrossRefGoogle Scholar
  17. 17.
    Husted, T., Massol, V.: JUnit in Action. Manning Publications Co. (2003)Google Scholar
  18. 18.
    Laddad, R.: AspectJ in Action: Practical Aspect-Oriented Programming. Manning (2003)Google Scholar
  19. 19.
    Mackinnon, T., Freeman, S., Craig, P.: Endo-testing: unit testing with mock objects. In: Extreme Programming Examined, pp. 287–301. Addison-Wesley, Reading (2001)Google Scholar
  20. 20.
    Pasternak, B., Tyszberowicz, S., Yehudai, A.: GenUTest: a unit test and mock aspect generation tool. Journal on Software Tools for Technology Transfer (2009)Google Scholar
  21. 21.
    Saff, D., Artzi, S., Perkins, J.H., Ernst, M.D.: Automatic test factoring for java. In: Proceedings, Automated Software Engineering, pp. 114–123 (2005)Google Scholar
  22. 22.
    Smaragdakis, Y., Csallner, C.: Combining static and dynamic reasoning for bug detection. In: Gurevich, Y., Meyer, B. (eds.) TAP 2007. LNCS, vol. 4454, pp. 1–16. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  23. 23.
    Strichman, O.: Regression verification: Proving the equivalence of similar programs. In: Bouajjani, A., Maler, O. (eds.) CAV 2009. LNCS, vol. 5643, p. 63. Springer, Heidelberg (2009)Google Scholar
  24. 24.
    Taneja, K., Xie, T.: DiffGen: Automated regression unit-test generation. In: Proceedings, Automated Software Engineering (2008)Google Scholar
  25. 25.
    Tillmann, N., de Halleux, J.: Pex-white box test generation for .NET. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 134–153. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  26. 26.
    van Vliet, H.: Software Engineering: Principles and Practice, 2nd edn. John Wiley & Sons, Inc., Chichester (2000)Google Scholar
  27. 27.
    Visser, W., Pasareanu, C.S., Khurshid, S.: Test input generation with Java PathFinder. In: Avrunin, G.S., Rothermel, G. (eds.) Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2004, Boston, Massachusetts, USA, July 11–14, pp. 97–107. ACM, New York (2004)CrossRefGoogle Scholar
  28. 28.
    Xie, T.: Augmenting automatically generated unit-test suites with regression oracle checking. In: Thomas, D. (ed.) ECOOP 2006. LNCS, vol. 4067, pp. 380–403. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  29. 29.
    Xie, T., Marinov, D., Notkin, D.: Rostra: A framework for detecting redundant object-oriented unit tests. In: Proceedings, Automated Software Engineering, pp. 196–205 (2004)Google Scholar
  30. 30.
    AspectJ, http://www.eclipse.org/aspectj (Visited January 2010)
  31. 31.
    Extreme Programming, http://www.extremeprogramming.org (Visited January 2010)

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Christoph Gladisch
    • 1
  • Shmuel Tyszberowicz
    • 2
  • Bernhard Beckert
    • 3
  • Amiram Yehudai
    • 4
  1. 1.Department of Computer ScienceUniversity of Koblenz 
  2. 2.School of Computer ScienceThe Academic College of Tel Aviv Yaffo 
  3. 3.Institut für Theoretische InformatikKarlsruhe Institute of Technology 
  4. 4.School of Computer ScienceTel Aviv University 

Personalised recommendations