A Simple and Practical Approach to Unit Testing: The JML and JUnit Way

  • Yoonsik Cheon
  • Gary T. Leavens
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2374)

Abstract

Writing unit test code is labor-intensive, hence it is often not done as an integral part of programming. However, unit testing is a practical approach to increasing the correctness and quality of software; for example, the Extreme Programming approach relies on frequent unit testing.

In this paper we present a new approach that makes writing unit tests easier. It uses a formal specification language’s runtime assertion checker to decide whether methods are working correctly, thus automating the writing of unit test oracles. These oracles can be easily combined with hand-written test data. Instead of writing testing code, the programmer writes formal specifications (e.g., pre-and postconditions). This makes the programmer’s task easier, because specifications are more concise and abstract than the equivalent test code, and hence more readable and maintainable. Furthermore, by using specifications in testing, specification errors are quickly discovered, so the specifications are more likely to provide useful documentation and inputs to other tools. We have implemented this idea using the Java Modeling Language (JML) and the JUnit testing framework, but the approach could be easily implemented with other combinations of formal specification languages and unit test tools.

Keywords

Unit Test Test Class Test Input Test Failure Java Modeling Language 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Karine Arnout and Raphael Simon. The.NET contract wizard: Adding design by contract to languages other than Eiffel. In Proceedings of TOOLS 39, 29 July–3 August 2001, Santa Barbara, California, pages 14–23. IEEE Computer Society, 2001.Google Scholar
  2. 2.
    D. Bartetzko, C. Fischer, M. Moller, and H. Wehrheim. Jass-Java with assertions. In Workshop on Runtime Verification held in conjunction with the 13th Conference on Computer Aided Verification, CAV’01, 2001.Google Scholar
  3. 3.
    Kent Beck and Erich Gamma. Test infected: Programmers love writing tests. Java Report, 3(7), July 1998.Google Scholar
  4. 4.
    Kent Beck. Extreme Programming Explained. Addison-Wesley, 2000.Google Scholar
  5. 5.
    Gilles Bernot, Marie Claude Claudel, and Bruno Marre. Software testing based on formal specifications: a theory and a tool. Software Engineering Journal, 6(6):387–405, November 1991.Google Scholar
  6. 6.
    Abhay Bhorkar. A run-time assertion checker for Java using JML. Technical Report TR #00-08, Department of Computer Science; Iowa State University, Ames, IA, May 2000.Google Scholar
  7. 7.
    Manuela Carrillo-Castellon, Jesus Garcia-Molina, Ernesto Pimentel, and Israel Repiso. Design by contract in Smalltalk. Journal of Object-Oriented Programming, 9(7):23–28, November/December 1996.Google Scholar
  8. 8.
    Juei Chang, Debra J. Richardson, and Sriram Sankar. Structural specification-based testing with ADL. In Proceedings of ISSTA 96, San Diego, CA, pages 62–70. IEEE Computer Society, 1996.Google Scholar
  9. 9.
    Yoonsik Cheon and Gary T. Leavens. A runtime assertion checker for the Java Modeling Language (JML). Technical Report 02-05, Department of Computer Science, Iowa State University, March 2002.Google Scholar
  10. 10.
    J. L. Crowley, J. F. Leathrum, and K. A. Liburdy. Isues in the full scale use of formal methods for automated testing. ACM SIGSOFT Software Engineering Notes, 21(3):71–78, May 1996.Google Scholar
  11. 11.
    David L. Detlefs, K. Rustan M. Leino, Greg Nelson, and James B. Saxe. Extended static checking. SRC Research Report 159, Compaq Systems Research Center, 130 Lytton Ave., Palo Alto, Dec 1998.Google Scholar
  12. 12.
    Carolyn K. Duby, Scott Meyers, and Steven P. Reiss. CCEL: A metalanguage for C++. In USENIX C++ Technical Conference Proceedings, pages 99–115, Portland, OR, August 1992. U ENIX Assoc. Berkeley, CA, USA.Google Scholar
  13. 13.
    Andrew Duncan and Urs Holzle. Adding contracts to Java with Handshake. Technical Report TRCS98-32, Department of Computer Science, University of California, Santa Barbara, CA, December 1998.Google Scholar
  14. 14.
    Robert Bruce Findler and Matthias Felleisen. Behavioral interface contracts for Java. Technical Report CS TR00-366, Department of Computer Science, Rice University, Houston, TX, August 2000.Google Scholar
  15. 15.
    Pedro Guerreiro. Simple support for design by contract in C++. In Proceedings of TOOLS 39, 29 July–3 August 2001, Santa Barbara, California, pages 24–34. IEEE Computer Society, 2001.Google Scholar
  16. 16.
    R. G. Hamlet. Testing programs with the aid of a compiler. IEEE Transactions on Software Engineering, 3(4):279–290, July 1977.Google Scholar
  17. 17.
    Teruo Higashino and Gregor v. Bochmann. Automatic analysis and test case derivation for a restricted class of LOTOS expressions with data parameters. IEEE Transactions on Software Engineering, 20(1):29–42, January 1994.Google Scholar
  18. 18.
    Bart Jacobs and Eric Poll. A logic for the Java modeling language JML. In Fundamental Approaches to Software Engineering (FASE’2001), Genova, Italy, 2001, volume 2029 of Lecture Notes in Computer Science, pages 284–299. Springer-Verlag, 2001.CrossRefGoogle Scholar
  19. 19.
    Pankaj Jalote. Specification and testing of abstract data types. Computing Languages, 17(1):75–82, 1992.CrossRefGoogle Scholar
  20. 20.
  21. 21.
    Murat Karaorman, Urs Holzle, and John Bruno. jContractor: A reflective Java library to support design by contract. In Pierre Cointe, editor, Meta-Level Architectures and Reflection, Second International Conference on Reflection’ 99, Saint-Malo, France, July 19–21, 1999, Proceedings, volume 1616 of Lecture Notes in Computer Science, pages 175–196. Springer-Verlag, July 1999.Google Scholar
  22. 22.
    Bogdan Korel and Ali M. Al-Yami. Automated regression test generation. In Proceedings of ISSTA 98, Clearwater Beach, FL, pages 143–152. IEEE Computer Society, 1998.Google Scholar
  23. 23.
    Reto Kramer. iContract-the Java design by contract tool. TOOLS 26: Technology of Object-Oriented Kanguages and Systems, Los Alamitos, California, pages 295–307, 1998.Google Scholar
  24. 24.
    Gary T. Leavens, Albert L. Baker, and Clyde Ruby. Preliminary design of JML: A behavioral interface specification language for Java. Technical Report 98–06p, Iowa State University, Department of Computer Science, August 2001. See http://www.jmlspecs.org.
  25. 25.
    Gary T. Leavens, Albert L. Baker, and Clye Ruby. JML: A notation for detailed design. In Haim Kilov, Bernhard Rumpe, and Ian Simmonds, editors, Behavioral Specifications of Businesses and Systems, chapter 12, pages 175–188. Kluwer, 1999.Google Scholar
  26. 26.
    Gary T. Leavens and Albert L. Baker. Enhancing the pre-and postcondition technique for more expressive specifications. In J. Davies J.M. Wing, J. Woodcock, editor, FM’99-Formal Methods, World Congress on Formal Methods in the Development of Computing Systems, Toulouse, France, September 1999. Proceedings, Volume II, volume 1708 of Lecture Notes in Computer Science, pages 1087–1106. Springer-Verlag, September 1999.Google Scholar
  27. 27.
    Bertrand Meyer. Eiffel: The Language. Object-Oriented Series. Prentice Hall, New York, N.Y., 1992.MATHGoogle Scholar
  28. 28.
    Bertrand Meyer. Object-oriented Software Construction. Prentice Hall, New York, N.Y., second edition, 1997.MATHGoogle Scholar
  29. 29.
    B. Meyer. Applying design by contract. IEEE Computer, 25(10):40–51, October 1992.Google Scholar
  30. 30.
    D.J. Panzl. Automatic software test driver. IEEE Computer, pages 44–50, April 1978.Google Scholar
  31. 31.
    Dennis Peters and David L. Parnas. Generating a test oracle from program documentation. In Proceedings of ISSTA 94, Seattle, Washington, August, 1994, pages 58–65. IEEE Computer Society, August 1994.Google Scholar
  32. 32.
    Reinhold Plosch and Josef Pichler. Contracts: From analysis to C++ implementation. In Proceedings of TOOLS 30, pages 248–257. IEEE Computer Society, 1999.Google Scholar
  33. 33.
    Sara Porat and Paul Fertig. Class assertions in C++. Journal of Object-Oriented Programming, 8(2):30–37, May 1995.Google Scholar
  34. 34.
    Debra J. Richardson. TAOS: Testing with analysis and oracle support. In Proceedings of ISSTA 94, Seattle, Washington, August, 1994, pages 138–152. IEEE Computer Society, August 1994.Google Scholar
  35. 35.
    David R. Rosenblum. A practical approach to programming with assertions. IEEE Transactions on Software Engineering, 21(1):19–31, January 1995.Google Scholar
  36. 36.
    Sriram Sankar and Roger Hayes. ADL: An interface definition language for specifying and testing software. ACM SIGPLAN Notices, 29(8):13–21, August 1994. Proceedings of the Workshop on Interface Definition Language, Jeannette M. Wing (editor), Portland, Oregon.Google Scholar
  37. 37.
    P. Stocks and D. Carrington. Test template framework: A specification-based test case study. In Proceedings of the 1993 International Symposium on Software Testing and Analysis (ISSTA), pages 11–18. IEEE Computer Society, June 1993.Google Scholar
  38. 38.
    David Welch and Scott Strong. An exception-based assertion mechanism for C++. Journal of Object-Oriented Programming, 11(4):50–60, July/August 1998.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Yoonsik Cheon
    • 1
  • Gary T. Leavens
    • 1
  1. 1.Department of Computer ScienceIowa State UniversityAmesUSA

Personalised recommendations