Integrating Software Testing and Run-Time Checking in an Assertion Verification Framework

  • Edison Mera
  • Pedro Lopez-García
  • Manuel Hermenegildo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5649)


We present a framework that unifies unit testing and run-time verification (as well as static verification and static debugging). A key contribution of our overall approach is that we preserve the use of a unified assertion language for all of these tasks. We first describe a method for compiling run-time checks for (parts of) assertions which cannot be verified at compile-time via program transformation. This transformation allows checking preconditions and postconditions, including conditional postconditions, properties at arbitrary program points, and certain computational properties. Most importantly, we propose a minimal addition to the assertion language which allows defining unit tests to be run in order to detect possible violations of the (partial) specifications expressed by the assertions. We have implemented the framework within the Ciao/CiaoPP system and effectively applied it to the verification of ISO Prolog compliance and to the detection of different types of bugs in the Ciao system source code. Experimental results are presented that illustrate different trade-offs among program size, running time, or levels of verbosity of the messages shown to the user.


run-time verification unit testing static/dynamic debugging assertions program verification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Belli, F., Jack, O.: Implementation-based Analysis and Testing of Prolog Programs. In: ISSTA 1993: Proc. of the ACM SIGSOFT Int’l. Symp. on Software Testing and Analysis, pp. 70–80. ACM, New York (1993)CrossRefGoogle Scholar
  2. 2.
    Bueno, F., Cabeza, D., Carro, M., Hermenegildo, M., López-García, P., Puebla, G. (eds.): The Ciao System. Ref. Manual (v1.13). Technical report, C. S. School, UPM (2006),
  3. 3.
    Bueno, F., Deransart, P., Drabent, W., Ferrand, G., Hermenegildo, M., Maluszynski, J., Puebla, G.: On the Role of Semantic Approximations in Validation and Diagnosis of Constraint Logic Programs. In: Proc. of the 3rd. Int’l WS on Automated Debugging–AADEBUG, May 1997, pp. 155–170. U. Linköping Press (1997)Google Scholar
  4. 4.
    The CLIP Group. Program Assertions. The Ciao System Documentation Series – TR CLIP4/97.1, Facultad de Informática, UPM (August 1997)Google Scholar
  5. 5.
    Eickelmann, N.S., Richardson, D.J.: An Evaluation of Software Test Environment Architectures. In: ICSE 1996: Proc. of the Int’l. Conf. on Software Engineering, pp. 353–364. IEEE Computer Society, Los Alamitos (1996)Google Scholar
  6. 6.
    Gómez-Zamalloa, M., Albert, E., Puebla, G.: On the Generation of Test Data for Prolog by Partial Evaluation. In: Workshop on Logic-based methods in Programming Environments (WLPE 2008), vol. WLPE/2008/06, pp. 26–43 (2008)Google Scholar
  7. 7.
    Hermenegildo, M., Puebla, G., Bueno, F.: Using Global Analysis, Partial Specifications, and an Extensible Assertion Language for Program Validation and Debugging. In: The Logic Programming Paradigm: a 25–Year Perspective, pp. 161–192. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  8. 8.
    Hermenegildo, M., Puebla, G., Bueno, F., López García, P.: Integrated Program Debugging, Verification, and Optimization Using Abstract Interpretation (and The Ciao System Preprocessor). Science of Comp. Progr. 58(1–2) (2005)Google Scholar
  9. 9.
    Mera, E., López-García, P., Hermenegildo, M.: Towards Integrating Run-Time Checking and Software Testing in a Verification Framework. Technical Report CLIP1/2009.0, T. U. Madrid (UPM) (March 2009)Google Scholar
  10. 10.
    Puebla, G., Bueno, F., Hermenegildo, M.: An Assertion Language for Debugging of Constraint Logic Programs. In: ILPS 1997 WS on Tools and Environments for (C)LP (October 1997),
  11. 11.
    Puebla, G., Bueno, F., Hermenegildo, M.: A Framework for Assertion-based Debugging in Constraint Logic Programming. In: Bossi, A. (ed.) LOPSTR 1999. LNCS, vol. 1817. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  12. 12.
    Puebla, G., Bueno, F., Hermenegildo, M.: A Generic Preprocessor for Program Validation and Debugging. In: Deransart, P., Małuszyński, J. (eds.) DiSCiPl 1999. LNCS, vol. 1870, pp. 63–107. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Puebla, G., Bueno, F., Hermenegildo, M.: An Assertion Language for Constraint Logic Programs. In: Deransart, P., Małuszyński, J. (eds.) DiSCiPl 1999. LNCS, vol. 1870, pp. 23–61. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  14. 14.
    Puebla, G., Hermenegildo, M.: Implementation of Multiple Specialization in Logic Programs. In: Proc. ACM SIGPLAN Symposium on Partial Evaluation and Semantics Based Program Manipulation, June 1995, pp. 77–87. ACM Press, New York (1995)Google Scholar
  15. 15.
    Puebla, G., Hermenegildo, M.: Abstract Multiple Specialization and its Application to Program Parallelization. JLP 41(2&3), 279–316 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Wielemaker, J.: SWI Prolog Unit Tests,
  17. 17.
    Zhao, L., Gu, T., Qian, J., Cai, G.: Test Frame Updating in CPM Testing of Prolog Programs. Software Quality Control 16(2), 277–298 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Edison Mera
    • 1
  • Pedro Lopez-García
    • 2
    • 3
  • Manuel Hermenegildo
    • 2
    • 4
  1. 1.Complutense University of Madrid (UCM)Spain
  2. 2.IMDEA SoftwareSpain
  3. 3.Spanish Research Council (CSIC)Spain
  4. 4.School of Computer ScienceTechnical University of Madrid (UPM)Spain

Personalised recommendations