Abstract
This paper presents a theory of testing that integrates into Hoare and He’s Unifying Theory of Programming (UTP). We give test cases a denotational semantics by viewing them as specification predicates. This reformulation of test cases allows for relating test cases via refinement to specifications and programs. Having such a refinement order that integrates test cases, we develop a testing theory for fault-based testing.
Fault-based testing uses test data designed to demonstrate the absence of a set of pre-specified faults. A well-known fault-based technique is mutation testing. In mutation testing, first, faults are injected into a program by altering (mutating) its source code. Then, test cases that can detect these errors are designed. The assumption is that other faults will be caught, too. In this paper, we apply the mutation technique to both, specifications and programs.
Using our theory of testing, two new test case generation laws for detecting injected (anticipated) faults are presented: one is based on the semantic level of UTP design predicates, the other on the algebraic properties of a small programming language.
Similar content being viewed by others
References
Abrial J-R (1996) The B Book, assigning programs to meanings. Cambridge University Press, Cambridge
Aichernig BK, Delgado CC (2006) From faults via test purposes to test cases: on the fault-based testing of concurrent systems. In: Baresi L, Heckel R (eds) Proceedings of FASE’06, Fundamental Approaches to Software Engineering, Vienna, Austria, March 27–29, 2006. Lecture Notes in Computer Science, vol 3922. Springer, Berlin, pp 324–338
Aichernig BK (2001) Systematic black-box testing of computer-based systems through formal abstraction techniques. Ph.D. thesis, Institute for Software Technology, TU Graz, Austria, January 2001 Supervisor: Peter Lucas
Aichernig BK (2001) Test-design through abstraction: a systematic approach based on the refinement calculus. J Universal Comput Sci 7(8): 710–735
Aichernig BK (2003) Mutation testing in the refinement calculus. Formal Asp Comput J 15(2): 280–295
Aichernig BK, Antonio Pari Salas P (2005) Test case generation by OCL mutation and constraint solving. In: Cai K-Y, Ohnishi A, Lau MF(eds) QSIC 2005, Proceedings of the Fifth International Conference on Quality Software, Melbourne, Australia, September 19–21, 2005. IEEE Computer Society Press, New York, pp 64–71
Benattou M, Bruel J-M, Hameurlain N (2007) Generating Test Data from OCL Specifications (position paper). In: ECOOP’2002 Workshop on Integration and Transformation of UML Models (WITUML02), Malaga, Spain, June 2002. http://ctp.di.fct.unl.pt/~ja/wituml/bruel.pdf (last visited Sep. 1st, 2007)
Beizer B (1990) Software testing techniques, 2nd edn. Van Nostrand Reinhold, New York
Budd TA, Gopal AS (1985) Program testing by specification mutation. Comput Lang 10(1): 63–73
Black PE, Okun V, Yesha Y (2001) Mutation of model checker specifications for test generation and evaluation. In: Mutation testing for the new century. Kluwer, Dordrecht, pp 14–20
Burton S (2000) Automated testing from Z specifications. Technical report YCS 329, Department of Computer Science University of York
Back R-J, von Wright J (1998) Refinement Calculus, a Systematic Introduction. Graduate Texts in Computer Science. Springer, Berlin
Dick J, Faivre A (1993) Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock JCP, Larsen PG (eds) Proceedings of FME’93: Industrial-Strength Formal Methods, International Symposium of Formal Methods Europe, April 1993, Odense, Denmark. Springer, Berlin, pp 268–284
DeMillo R, Lipton R, Sayward F (1978) Hints on test data selection: Help for the practicing programmer. IEEE Comput 11(4): 34–41
De Nicola R, Hennessy MCB (1984) Testing equivalences for processes. Theor Comput Sci 34: 83–133
Frühwirth T, Abdennadher S (2003) Essentials of constraint programming. Springer, Berlin
Fraser G, Aichernig BK, Wotawa F (2007) Handling model changes: Regression testing and test-suite update with model-checkers. Electronic Notes in Theoretical Computer Science, vol 190. Elsevier, Amsterdam, pp 33–46
Gaudel M-C (1995) Testing can be formal, too. In: TAPSOFT ’95: Proceedings of the 6th International Joint Conference CAAP/FASE on Theory and Practice of Software Development. Springer, Berlin, pp 82–96
Gaudel M-C, James PR (1998) Testing algebraic data types and processes: a unifying theory. Formal Asp Comput 10(5, 6): 436–451
Hamlet RG (1977) Testing programs with the aid of a compiler. IEEE Trans Softw Eng 3(4): 279–290
Hoare CAR, He J (1998) Unifying theories of programming. Prentice-Hall, Englewood Cliffs, NJ
Helke S, Neustupny T, Santen T (1997) Automating test case generation from Z specifications with Isabelle. In: Bowen JP, Hinchey MG, Till D (eds) Proceedings of ZUM’97, the 10th International Conference of Z Users, April 1997, Reading, UK. Lecture Notes in Computer Science, vol 1212. Springer, Berlin, pp 52–71
Hoare CAR (1985) Communicating sequential processes. International Series in Computer Science. Prentice-Hall, Englewood Cliffs, NJ
Jackson D (2000) Automating first-order relational logic. In: Proceedings of SIGSOFT FSE 2000: ACM SIGSOFT Symposium on Foundations of Software Engineering, November 6–10, 2000, San Diego, California, USA. ACM, New York, pp 130–139
Jones CB (1986) Systematic software development using VDM. International Series in Computer Science. Prentice-Hall, Englewood Cliffs, NJ
Jorgensen PC (2002) Software testing: a craftsman’s approach, 2nd edn. CRC Press, Boca Raton, FL
Legeard B, Peureux F, Utting M (2002) Automated boundary testing from Z and B. In: Eriksson L-H, Lindsay PA (eds) Proceedings of FME 2002: Formal Methods—Getting IT Right, International Symposium of Formal Methods Europe, July 2002, Copenhagen, Denmark. Lecture Notes in Computer Science, vol 2391. Springer, Berlin, pp 21–40
Morgan C (1994) Programming from specifications, 2nd edn. International Series In Computer Science. Prentice-Hall, Englewood Cliffs, NJ
Peleska J, Siegel M (1996) From testing theory to test driver implementation. In: Gaudel M-C, Woodcock J (eds) FME’96: Industrial Benefit and Advances in Formal Methods, March 1996, Springer-Verlag, pp 538–556
RAISE Development Group (1995) The RAISE development method. Prentice-Hall, UK
Srivatanakul T, Clark J, Stepney S, Polack F (2003) Challenging formal specifications by mutation: a CSP security example. In: Proceedings of APSEC-2003: 10th Asia-Pacific Software Engineering Conference, Chiang Mai, Thailand, December, 2003. IEEE, New York, pp 340–351
Simone do Rocio Senger de Souza, Jose Carlos Maldonado, Sandra Camargo Pinto Ferraz Fabbri, Wanderley Lopes de Souza (1999) Mutation testing applied to Estelle specifications. Softw Q J 8:285–301
IEEE Computer Society (1990) Standard glossary of software engineering terminology, Standard 610.12. IEEE Press, New York
Srivatanakul T (2001) Mutation testing for concurrency. Master’s thesis, Department of Computer Science, University of York, UK, September 2001
Stepney S (1995) Testing as abstraction. In: Bowen JP, Hinchey MG (eds) ZUM ’95: 9th International Conference of Z Users, Limerick 1995. Lecture Notes in Computer Science, vol 967. Springer, Berlin, pp 137–151
Stocks PA (1993) Applying formal methods to software testing. Ph.D. thesis, Department of computer science, University of Queensland
Tai K-C (1996) Theory of fault-based predicate testing for computer programs. IEEE Trans Softw Eng 22(8): 552–562
Tretmans J (1992) A formal approach to conformance testing. Ph.D. thesis, Universiteit Twente
Tretmans J (1999) Testing concurrent systems: A formal approach. In: Baeten JCM, Mauw S (eds) CONCUR’99. Lecture Notes in Computer Science, vol 1664. Springer, Berlin, pp 46–65
Tai K-C, Su H-K (1987) Test generation for Boolean expressions. In: Proceedings of the Eleventh Annual International Computer Software and Applications Conference (COMPSAC), pp 278–284
Wimmel G, Jürjens J (2002) Specification-based test generation for security-critical systems using mutations. In: George C, Huaikou M (eds) Proceedings of ICFEM’02, the International Conference of Formal Engineering Methods, October 21–25, 2002, Shanghai, China, Lecture Notes in Computer Science. Springer, Berlin, pp 471–482
Warmer J, Kleppe A (2003) The object constraint language: getting your models ready for MDA, 2nd edn. Addison-Wesley, Reading, MA
Eric Wong, W (eds) (2001) Mutation Testing for the New Century. Kluwer, Dordrecht
Author information
Authors and Affiliations
Corresponding author
Additional information
E. A. Boiten, M. J. Butler, J. Derrick, L. Groves and J. C. P. Woodcock
Rights and permissions
About this article
Cite this article
Aichernig, B.K., Jifeng, H. Mutation testing in UTP. Form Asp Comp 21, 33–64 (2009). https://doi.org/10.1007/s00165-008-0083-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00165-008-0083-6