Abstract
This article presents a new formal approach to testing. In the field of dynamic testing, as soon as a program fails for a test set, it is flagged incorrect. The remaining question is: how far can a successful program be considered as correct? We give a definition of program correctness with respect to a specification which is adequate to dynamic testing. Similarly to the field of abstract implementation, the idea is that in order to declare a program as correct, it suffices that its behavior fulfills the specification requirements. An intermediate semantic level between the program and the specification, called the oracle framework, is introduced in order to interpret observable results obtained from dynamic experiments on the program. This allows to give algebraic semantics (i.e. a set of models) to the program, compatible with the program behavior. Program correctness is then defined by some adequacy criterion between the specification semantics and the program semantics. We point out that while for some specifications, there exist exhaustive test sets (the success of which means program correctness), for some other specifications, there only exist “complete” (but not exhaustive) test sets. Of course, all the programs rejected by a complete test set are incorrect but unfortunately, there still exist successful incorrect programs. We also explain how the test set selection can be formalized within our approach.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
M. Aiguier and G Bernot. Algebraic semantics of object type specifications. In R.J.Wieringa R.B.Feenstra eds, editor, Information Systems Correctness and Reusability, Selected papers from the IS-CORE workshop. World Scientific Publishing, 1995.
G. Bernot. Testing against formal specifications: a theoretical view. In TAPSOFT'91, Internationnal Joint Conference on the Theory and Practice of Software Developement, number 494 in Springer Verlag, LNCS, pages 99–119, 1991.
G. Bernot, M.C. Gaudel, and B. Marre. Software testing based on formal specifications: a theory and a tool. Software Engineering Journal, Vol.6(6):387–405, 1991. (also technical report LRI 581, Orsay (Avril 1991)).
G. Bernot and P. Le Gall. Exception handling and term labelling. In TAPSOFT'93, Internationnal Joint Conference, Theory And Practice of Software Developement, number 668 in Springer Verlag, LNCS, pages 421–436, 1993.
M. Bidoit, R. Hennicker, and M. Wirsing. Behavioural and abstractor specifications. Science of Computer Programming, 25:149–186, 1995.
E. Brinksma, J. Tretmans, and L. Verhaard. A framework for test selection. In Procol Specification, Testing, and Verification XI, North Holland, 1991.
M. Cerioli and G. Reggio. Institutions for very abstract specifications. In LNCS, editor, Recent Trends in Data Type Specification, volume 785, pages 113–127, 1994.
P. Dauchy, M.-C. Gaudel, and B. Marre. Using algebraic specifications in software testing: a case study on the software of an automatic subway. Journal of Systems and Software, 21(3):229–244, 1993.
H. Ehrig and B. Mahr. Fundamentals of Algebraic Specification 1. Equations and initial semantics, volume 6. Springer-Verlag,EATCS Monographs on Theoretical Computer Science, 1985.
M.C. Gaudel. Testing can be formal, too. TAPSOFT'95, Theory and Practice of Software Developement, 6th International Joint Conference, CAAP/FASE, Aarhus, Denmark, LNCS 915, 1995.
J.A. Goguen and R.M. Burstall. Introducing institutions. In Springer-Verlag LNCS 164, editor, Proc. of the Workshop on Logics of Programming, pages 221–256, 1984.
R. Hennicker. Context induction: a proof principle for behavioural abstractions and algebraic implementations. Formal Aspects of Computing, 3(4):326–345, 1991.
P. Le Gall. Les algèbres étiquetées: une sémantique pour les spécifications algébriques fondée sur une utilisation systématique des termes. Application au test de logiciel avec traitement d'exceptions. Phd, Université de Paris XI, 1993.
B. Marre. Toward automatic test data set selection using Algebraic Specifications and Logic Programming. mEigth International Conference on Logic Programming, ICLP'91, Paris, 25–28, MIT Press, 1991.
B. Marre, P. Thevenod, H. Waeselynk, P. Le Gall, and Y. Crouset. An experimental evaluation of formal testing and statistical testing. In SAFECOMP'92, Actes Safety of Computer Control System 1992, Pergamon Press”, 1992.
M. Phalippou. Relations d'implantation et hypothèses de test sur des automates à entrées et sorties. Phd, Université de Bordeaux, 1994.
D.J. Richardson, S. Aha, and T. Owen O'Malley. Specification-based test oracles for reactive systems. In Proceedings of the 14th International Conference on Software Engineering, pages 105–118, 1992.
D. Sannella and A. Tarlecki. Toward formal development of programs from algebraic specification: implementation revisited. Acta Informatica, 25:233–281, 1988.
P. Stocks and D.A. Carrington. Test template framework: A specification-based testing framework. In Proceedings of the 15th International Conference on Software Engineering, pages 405–414, 1993.
E.J. Weyuker. On testing non testable programs. In The Computer Journal 25, pages 465–470, 1982.
M. Wirsing. Algebraic Specification. Handbook of Theoretical Computer Science, vol B, the MIT Press, 1990.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Le Gall, P., Arnould, A. (1996). Formal specifications and test: Correctness and oracle. In: Haveraaen, M., Owe, O., Dahl, OJ. (eds) Recent Trends in Data Type Specification. ADT COMPASS 1995 1995. Lecture Notes in Computer Science, vol 1130. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-61629-2_52
Download citation
DOI: https://doi.org/10.1007/3-540-61629-2_52
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-61629-0
Online ISBN: 978-3-540-70642-7
eBook Packages: Springer Book Archive