A Simple and Practical Approach to Unit Testing: The JML and JUnit Way
Writing unit test code is labor-intensive, hence it is often not done as an integral part of programming. However, unit testing is a practical approach to increasing the correctness and quality of software; for example, the Extreme Programming approach relies on frequent unit testing.
In this paper we present a new approach that makes writing unit tests easier. It uses a formal specification language’s runtime assertion checker to decide whether methods are working correctly, thus automating the writing of unit test oracles. These oracles can be easily combined with hand-written test data. Instead of writing testing code, the programmer writes formal specifications (e.g., pre-and postconditions). This makes the programmer’s task easier, because specifications are more concise and abstract than the equivalent test code, and hence more readable and maintainable. Furthermore, by using specifications in testing, specification errors are quickly discovered, so the specifications are more likely to provide useful documentation and inputs to other tools. We have implemented this idea using the Java Modeling Language (JML) and the JUnit testing framework, but the approach could be easily implemented with other combinations of formal specification languages and unit test tools.
Unable to display preview. Download preview PDF.
- 1.Karine Arnout and Raphael Simon. The.NET contract wizard: Adding design by contract to languages other than Eiffel. In Proceedings of TOOLS 39, 29 July–3 August 2001, Santa Barbara, California, pages 14–23. IEEE Computer Society, 2001.Google Scholar
- 2.D. Bartetzko, C. Fischer, M. Moller, and H. Wehrheim. Jass-Java with assertions. In Workshop on Runtime Verification held in conjunction with the 13th Conference on Computer Aided Verification, CAV’01, 2001.Google Scholar
- 3.Kent Beck and Erich Gamma. Test infected: Programmers love writing tests. Java Report, 3(7), July 1998.Google Scholar
- 4.Kent Beck. Extreme Programming Explained. Addison-Wesley, 2000.Google Scholar
- 5.Gilles Bernot, Marie Claude Claudel, and Bruno Marre. Software testing based on formal specifications: a theory and a tool. Software Engineering Journal, 6(6):387–405, November 1991.Google Scholar
- 6.Abhay Bhorkar. A run-time assertion checker for Java using JML. Technical Report TR #00-08, Department of Computer Science; Iowa State University, Ames, IA, May 2000.Google Scholar
- 7.Manuela Carrillo-Castellon, Jesus Garcia-Molina, Ernesto Pimentel, and Israel Repiso. Design by contract in Smalltalk. Journal of Object-Oriented Programming, 9(7):23–28, November/December 1996.Google Scholar
- 8.Juei Chang, Debra J. Richardson, and Sriram Sankar. Structural specification-based testing with ADL. In Proceedings of ISSTA 96, San Diego, CA, pages 62–70. IEEE Computer Society, 1996.Google Scholar
- 9.Yoonsik Cheon and Gary T. Leavens. A runtime assertion checker for the Java Modeling Language (JML). Technical Report 02-05, Department of Computer Science, Iowa State University, March 2002.Google Scholar
- 10.J. L. Crowley, J. F. Leathrum, and K. A. Liburdy. Isues in the full scale use of formal methods for automated testing. ACM SIGSOFT Software Engineering Notes, 21(3):71–78, May 1996.Google Scholar
- 11.David L. Detlefs, K. Rustan M. Leino, Greg Nelson, and James B. Saxe. Extended static checking. SRC Research Report 159, Compaq Systems Research Center, 130 Lytton Ave., Palo Alto, Dec 1998.Google Scholar
- 12.Carolyn K. Duby, Scott Meyers, and Steven P. Reiss. CCEL: A metalanguage for C++. In USENIX C++ Technical Conference Proceedings, pages 99–115, Portland, OR, August 1992. U ENIX Assoc. Berkeley, CA, USA.Google Scholar
- 13.Andrew Duncan and Urs Holzle. Adding contracts to Java with Handshake. Technical Report TRCS98-32, Department of Computer Science, University of California, Santa Barbara, CA, December 1998.Google Scholar
- 14.Robert Bruce Findler and Matthias Felleisen. Behavioral interface contracts for Java. Technical Report CS TR00-366, Department of Computer Science, Rice University, Houston, TX, August 2000.Google Scholar
- 15.Pedro Guerreiro. Simple support for design by contract in C++. In Proceedings of TOOLS 39, 29 July–3 August 2001, Santa Barbara, California, pages 24–34. IEEE Computer Society, 2001.Google Scholar
- 16.R. G. Hamlet. Testing programs with the aid of a compiler. IEEE Transactions on Software Engineering, 3(4):279–290, July 1977.Google Scholar
- 17.Teruo Higashino and Gregor v. Bochmann. Automatic analysis and test case derivation for a restricted class of LOTOS expressions with data parameters. IEEE Transactions on Software Engineering, 20(1):29–42, January 1994.Google Scholar
- 20.JUnit. http://www.junit.org.
- 21.Murat Karaorman, Urs Holzle, and John Bruno. jContractor: A reflective Java library to support design by contract. In Pierre Cointe, editor, Meta-Level Architectures and Reflection, Second International Conference on Reflection’ 99, Saint-Malo, France, July 19–21, 1999, Proceedings, volume 1616 of Lecture Notes in Computer Science, pages 175–196. Springer-Verlag, July 1999.Google Scholar
- 22.Bogdan Korel and Ali M. Al-Yami. Automated regression test generation. In Proceedings of ISSTA 98, Clearwater Beach, FL, pages 143–152. IEEE Computer Society, 1998.Google Scholar
- 23.Reto Kramer. iContract-the Java design by contract tool. TOOLS 26: Technology of Object-Oriented Kanguages and Systems, Los Alamitos, California, pages 295–307, 1998.Google Scholar
- 24.Gary T. Leavens, Albert L. Baker, and Clyde Ruby. Preliminary design of JML: A behavioral interface specification language for Java. Technical Report 98–06p, Iowa State University, Department of Computer Science, August 2001. See http://www.jmlspecs.org.
- 25.Gary T. Leavens, Albert L. Baker, and Clye Ruby. JML: A notation for detailed design. In Haim Kilov, Bernhard Rumpe, and Ian Simmonds, editors, Behavioral Specifications of Businesses and Systems, chapter 12, pages 175–188. Kluwer, 1999.Google Scholar
- 26.Gary T. Leavens and Albert L. Baker. Enhancing the pre-and postcondition technique for more expressive specifications. In J. Davies J.M. Wing, J. Woodcock, editor, FM’99-Formal Methods, World Congress on Formal Methods in the Development of Computing Systems, Toulouse, France, September 1999. Proceedings, Volume II, volume 1708 of Lecture Notes in Computer Science, pages 1087–1106. Springer-Verlag, September 1999.Google Scholar
- 29.B. Meyer. Applying design by contract. IEEE Computer, 25(10):40–51, October 1992.Google Scholar
- 30.D.J. Panzl. Automatic software test driver. IEEE Computer, pages 44–50, April 1978.Google Scholar
- 31.Dennis Peters and David L. Parnas. Generating a test oracle from program documentation. In Proceedings of ISSTA 94, Seattle, Washington, August, 1994, pages 58–65. IEEE Computer Society, August 1994.Google Scholar
- 32.Reinhold Plosch and Josef Pichler. Contracts: From analysis to C++ implementation. In Proceedings of TOOLS 30, pages 248–257. IEEE Computer Society, 1999.Google Scholar
- 33.Sara Porat and Paul Fertig. Class assertions in C++. Journal of Object-Oriented Programming, 8(2):30–37, May 1995.Google Scholar
- 34.Debra J. Richardson. TAOS: Testing with analysis and oracle support. In Proceedings of ISSTA 94, Seattle, Washington, August, 1994, pages 138–152. IEEE Computer Society, August 1994.Google Scholar
- 35.David R. Rosenblum. A practical approach to programming with assertions. IEEE Transactions on Software Engineering, 21(1):19–31, January 1995.Google Scholar
- 36.Sriram Sankar and Roger Hayes. ADL: An interface definition language for specifying and testing software. ACM SIGPLAN Notices, 29(8):13–21, August 1994. Proceedings of the Workshop on Interface Definition Language, Jeannette M. Wing (editor), Portland, Oregon.Google Scholar
- 37.P. Stocks and D. Carrington. Test template framework: A specification-based test case study. In Proceedings of the 1993 International Symposium on Software Testing and Analysis (ISSTA), pages 11–18. IEEE Computer Society, June 1993.Google Scholar
- 38.David Welch and Scott Strong. An exception-based assertion mechanism for C++. Journal of Object-Oriented Programming, 11(4):50–60, July/August 1998.Google Scholar