Integrating Software Testing and Run-Time Checking in an Assertion Verification Framework
We present a framework that unifies unit testing and run-time verification (as well as static verification and static debugging). A key contribution of our overall approach is that we preserve the use of a unified assertion language for all of these tasks. We first describe a method for compiling run-time checks for (parts of) assertions which cannot be verified at compile-time via program transformation. This transformation allows checking preconditions and postconditions, including conditional postconditions, properties at arbitrary program points, and certain computational properties. Most importantly, we propose a minimal addition to the assertion language which allows defining unit tests to be run in order to detect possible violations of the (partial) specifications expressed by the assertions. We have implemented the framework within the Ciao/CiaoPP system and effectively applied it to the verification of ISO Prolog compliance and to the detection of different types of bugs in the Ciao system source code. Experimental results are presented that illustrate different trade-offs among program size, running time, or levels of verbosity of the messages shown to the user.
Keywordsrun-time verification unit testing static/dynamic debugging assertions program verification
Unable to display preview. Download preview PDF.
- 2.Bueno, F., Cabeza, D., Carro, M., Hermenegildo, M., López-García, P., Puebla, G. (eds.): The Ciao System. Ref. Manual (v1.13). Technical report, C. S. School, UPM (2006), http://www.ciaohome.org
- 3.Bueno, F., Deransart, P., Drabent, W., Ferrand, G., Hermenegildo, M., Maluszynski, J., Puebla, G.: On the Role of Semantic Approximations in Validation and Diagnosis of Constraint Logic Programs. In: Proc. of the 3rd. Int’l WS on Automated Debugging–AADEBUG, May 1997, pp. 155–170. U. Linköping Press (1997)Google Scholar
- 4.The CLIP Group. Program Assertions. The Ciao System Documentation Series – TR CLIP4/97.1, Facultad de Informática, UPM (August 1997)Google Scholar
- 5.Eickelmann, N.S., Richardson, D.J.: An Evaluation of Software Test Environment Architectures. In: ICSE 1996: Proc. of the Int’l. Conf. on Software Engineering, pp. 353–364. IEEE Computer Society, Los Alamitos (1996)Google Scholar
- 6.Gómez-Zamalloa, M., Albert, E., Puebla, G.: On the Generation of Test Data for Prolog by Partial Evaluation. In: Workshop on Logic-based methods in Programming Environments (WLPE 2008), vol. WLPE/2008/06, pp. 26–43 (2008)Google Scholar
- 8.Hermenegildo, M., Puebla, G., Bueno, F., López García, P.: Integrated Program Debugging, Verification, and Optimization Using Abstract Interpretation (and The Ciao System Preprocessor). Science of Comp. Progr. 58(1–2) (2005)Google Scholar
- 9.Mera, E., López-García, P., Hermenegildo, M.: Towards Integrating Run-Time Checking and Software Testing in a Verification Framework. Technical Report CLIP1/2009.0, T. U. Madrid (UPM) (March 2009)Google Scholar
- 10.Puebla, G., Bueno, F., Hermenegildo, M.: An Assertion Language for Debugging of Constraint Logic Programs. In: ILPS 1997 WS on Tools and Environments for (C)LP (October 1997), ftp://clip.dia.fi.upm.es/pub/papers-/assert_lang_tr_discipldeliv.ps.gz
- 14.Puebla, G., Hermenegildo, M.: Implementation of Multiple Specialization in Logic Programs. In: Proc. ACM SIGPLAN Symposium on Partial Evaluation and Semantics Based Program Manipulation, June 1995, pp. 77–87. ACM Press, New York (1995)Google Scholar
- 16.Wielemaker, J.: SWI Prolog Unit Tests, http://www.swi-prolog.org/pldoc/package/plunit.html