Skip to main content
Log in

Mutation testing in UTP

  • Original Article
  • Published:
Formal Aspects of Computing

Abstract

This paper presents a theory of testing that integrates into Hoare and He’s Unifying Theory of Programming (UTP). We give test cases a denotational semantics by viewing them as specification predicates. This reformulation of test cases allows for relating test cases via refinement to specifications and programs. Having such a refinement order that integrates test cases, we develop a testing theory for fault-based testing.

Fault-based testing uses test data designed to demonstrate the absence of a set of pre-specified faults. A well-known fault-based technique is mutation testing. In mutation testing, first, faults are injected into a program by altering (mutating) its source code. Then, test cases that can detect these errors are designed. The assumption is that other faults will be caught, too. In this paper, we apply the mutation technique to both, specifications and programs.

Using our theory of testing, two new test case generation laws for detecting injected (anticipated) faults are presented: one is based on the semantic level of UTP design predicates, the other on the algebraic properties of a small programming language.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abrial J-R (1996) The B Book, assigning programs to meanings. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  2. Aichernig BK, Delgado CC (2006) From faults via test purposes to test cases: on the fault-based testing of concurrent systems. In: Baresi L, Heckel R (eds) Proceedings of FASE’06, Fundamental Approaches to Software Engineering, Vienna, Austria, March 27–29, 2006. Lecture Notes in Computer Science, vol 3922. Springer, Berlin, pp 324–338

  3. Aichernig BK (2001) Systematic black-box testing of computer-based systems through formal abstraction techniques. Ph.D. thesis, Institute for Software Technology, TU Graz, Austria, January 2001 Supervisor: Peter Lucas

  4. Aichernig BK (2001) Test-design through abstraction: a systematic approach based on the refinement calculus. J Universal Comput Sci 7(8): 710–735

    Google Scholar 

  5. Aichernig BK (2003) Mutation testing in the refinement calculus. Formal Asp Comput J 15(2): 280–295

    Article  MATH  Google Scholar 

  6. Aichernig BK, Antonio Pari Salas P (2005) Test case generation by OCL mutation and constraint solving. In: Cai K-Y, Ohnishi A, Lau MF(eds) QSIC 2005, Proceedings of the Fifth International Conference on Quality Software, Melbourne, Australia, September 19–21, 2005. IEEE Computer Society Press, New York, pp 64–71

    Chapter  Google Scholar 

  7. Benattou M, Bruel J-M, Hameurlain N (2007) Generating Test Data from OCL Specifications (position paper). In: ECOOP’2002 Workshop on Integration and Transformation of UML Models (WITUML02), Malaga, Spain, June 2002. http://ctp.di.fct.unl.pt/~ja/wituml/bruel.pdf (last visited Sep. 1st, 2007)

  8. Beizer B (1990) Software testing techniques, 2nd edn. Van Nostrand Reinhold, New York

    Google Scholar 

  9. Budd TA, Gopal AS (1985) Program testing by specification mutation. Comput Lang 10(1): 63–73

    Article  MATH  Google Scholar 

  10. Black PE, Okun V, Yesha Y (2001) Mutation of model checker specifications for test generation and evaluation. In: Mutation testing for the new century. Kluwer, Dordrecht, pp 14–20

  11. Burton S (2000) Automated testing from Z specifications. Technical report YCS 329, Department of Computer Science University of York

  12. Back R-J, von Wright J (1998) Refinement Calculus, a Systematic Introduction. Graduate Texts in Computer Science. Springer, Berlin

    Google Scholar 

  13. Dick J, Faivre A (1993) Automating the generation and sequencing of test cases from model-based specifications. In: Woodcock JCP, Larsen PG (eds) Proceedings of FME’93: Industrial-Strength Formal Methods, International Symposium of Formal Methods Europe, April 1993, Odense, Denmark. Springer, Berlin, pp 268–284

  14. DeMillo R, Lipton R, Sayward F (1978) Hints on test data selection: Help for the practicing programmer. IEEE Comput 11(4): 34–41

    Google Scholar 

  15. De Nicola R, Hennessy MCB (1984) Testing equivalences for processes. Theor Comput Sci 34: 83–133

    Article  MATH  MathSciNet  Google Scholar 

  16. Frühwirth T, Abdennadher S (2003) Essentials of constraint programming. Springer, Berlin

    Google Scholar 

  17. Fraser G, Aichernig BK, Wotawa F (2007) Handling model changes: Regression testing and test-suite update with model-checkers. Electronic Notes in Theoretical Computer Science, vol 190. Elsevier, Amsterdam, pp 33–46

  18. Gaudel M-C (1995) Testing can be formal, too. In: TAPSOFT ’95: Proceedings of the 6th International Joint Conference CAAP/FASE on Theory and Practice of Software Development. Springer, Berlin, pp 82–96

  19. Gaudel M-C, James PR (1998) Testing algebraic data types and processes: a unifying theory. Formal Asp Comput 10(5, 6): 436–451

    Article  MATH  Google Scholar 

  20. Hamlet RG (1977) Testing programs with the aid of a compiler. IEEE Trans Softw Eng 3(4): 279–290

    Article  MathSciNet  Google Scholar 

  21. Hoare CAR, He J (1998) Unifying theories of programming. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  22. Helke S, Neustupny T, Santen T (1997) Automating test case generation from Z specifications with Isabelle. In: Bowen JP, Hinchey MG, Till D (eds) Proceedings of ZUM’97, the 10th International Conference of Z Users, April 1997, Reading, UK. Lecture Notes in Computer Science, vol 1212. Springer, Berlin, pp 52–71

  23. Hoare CAR (1985) Communicating sequential processes. International Series in Computer Science. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  24. Jackson D (2000) Automating first-order relational logic. In: Proceedings of SIGSOFT FSE 2000: ACM SIGSOFT Symposium on Foundations of Software Engineering, November 6–10, 2000, San Diego, California, USA. ACM, New York, pp 130–139

  25. Jones CB (1986) Systematic software development using VDM. International Series in Computer Science. Prentice-Hall, Englewood Cliffs, NJ

    Google Scholar 

  26. Jorgensen PC (2002) Software testing: a craftsman’s approach, 2nd edn. CRC Press, Boca Raton, FL

    Google Scholar 

  27. Legeard B, Peureux F, Utting M (2002) Automated boundary testing from Z and B. In: Eriksson L-H, Lindsay PA (eds) Proceedings of FME 2002: Formal Methods—Getting IT Right, International Symposium of Formal Methods Europe, July 2002, Copenhagen, Denmark. Lecture Notes in Computer Science, vol 2391. Springer, Berlin, pp 21–40

  28. Morgan C (1994) Programming from specifications, 2nd edn. International Series In Computer Science. Prentice-Hall, Englewood Cliffs, NJ

  29. Peleska J, Siegel M (1996) From testing theory to test driver implementation. In: Gaudel M-C, Woodcock J (eds) FME’96: Industrial Benefit and Advances in Formal Methods, March 1996, Springer-Verlag, pp 538–556

  30. RAISE Development Group (1995) The RAISE development method. Prentice-Hall, UK

    Google Scholar 

  31. Srivatanakul T, Clark J, Stepney S, Polack F (2003) Challenging formal specifications by mutation: a CSP security example. In: Proceedings of APSEC-2003: 10th Asia-Pacific Software Engineering Conference, Chiang Mai, Thailand, December, 2003. IEEE, New York, pp 340–351

  32. Simone do Rocio Senger de Souza, Jose Carlos Maldonado, Sandra Camargo Pinto Ferraz Fabbri, Wanderley Lopes de Souza (1999) Mutation testing applied to Estelle specifications. Softw Q J 8:285–301

  33. IEEE Computer Society (1990) Standard glossary of software engineering terminology, Standard 610.12. IEEE Press, New York

  34. Srivatanakul T (2001) Mutation testing for concurrency. Master’s thesis, Department of Computer Science, University of York, UK, September 2001

  35. Stepney S (1995) Testing as abstraction. In: Bowen JP, Hinchey MG (eds) ZUM ’95: 9th International Conference of Z Users, Limerick 1995. Lecture Notes in Computer Science, vol 967. Springer, Berlin, pp 137–151

  36. Stocks PA (1993) Applying formal methods to software testing. Ph.D. thesis, Department of computer science, University of Queensland

  37. Tai K-C (1996) Theory of fault-based predicate testing for computer programs. IEEE Trans Softw Eng 22(8): 552–562

    Article  Google Scholar 

  38. Tretmans J (1992) A formal approach to conformance testing. Ph.D. thesis, Universiteit Twente

  39. Tretmans J (1999) Testing concurrent systems: A formal approach. In: Baeten JCM, Mauw S (eds) CONCUR’99. Lecture Notes in Computer Science, vol 1664. Springer, Berlin, pp 46–65

  40. Tai K-C, Su H-K (1987) Test generation for Boolean expressions. In: Proceedings of the Eleventh Annual International Computer Software and Applications Conference (COMPSAC), pp 278–284

  41. Wimmel G, Jürjens J (2002) Specification-based test generation for security-critical systems using mutations. In: George C, Huaikou M (eds) Proceedings of ICFEM’02, the International Conference of Formal Engineering Methods, October 21–25, 2002, Shanghai, China, Lecture Notes in Computer Science. Springer, Berlin, pp 471–482

  42. Warmer J, Kleppe A (2003) The object constraint language: getting your models ready for MDA, 2nd edn. Addison-Wesley, Reading, MA

    Google Scholar 

  43. Eric Wong, W (eds) (2001) Mutation Testing for the New Century. Kluwer, Dordrecht

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bernhard K. Aichernig.

Additional information

E. A. Boiten, M. J. Butler, J. Derrick, L. Groves and J. C. P. Woodcock

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aichernig, B.K., Jifeng, H. Mutation testing in UTP. Form Asp Comp 21, 33–64 (2009). https://doi.org/10.1007/s00165-008-0083-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00165-008-0083-6

Keywords

Navigation