Programming and Computer Software

, Volume 33, Issue 5, pp 239–260 | Cite as

Formalization of test experiments

  • I. B. Bourdonov
  • A. S. Kossatchev
  • V. V. Kuliamin
Article

Abstract

Formal methods for testing conformance of the system under examination to its specification are examined. The operational interaction semantics is specified by a special testing machine that formally determines the testing capabilities. A set of theoretically powerful and practically important capabilities is distinguished that can be reduced to the observation of external actions and refusals (the absence of external actions). The novelties are as follows. (1) Parameterization of the semantics by the families of observable and not observable refusals, which makes it possible to take into account various constraints on the (correct) interactions. (2) Destruction as a forbidden action, which is possible but should not be performed in the case of a correct interaction. (3) Modeling of the divergence by the Δ-action, which also should be avoided in the case of a correct interaction. On the basis of this semantics, the concept of safe testing, the implementation safety hypothesis, and the safe conformance relation are proposed. The safe conformance relation corresponds to the principle of independent observations: a behavior of an implementation is correct or incorrect independently of its other possible behaviors. For a more narrow class of interactions, another version of the semantics based on the ready traces may be used along with the corresponding conformance relation. Some propositions concerning the relationships between the conformance relations under various semantics are formulated. The completion transformation that solves the problem of the conformance relation reflexivity and a monotone transformation that solves the monotonicity problem (preservation of the conformance under composition) are defined.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bernot, G., Testing against Formal Specifications: A Theoretical View, TAPSOFT’91, Abramsky, S. and Maibaum, T.S.F., Eds., vol. 2, pp. 99–119, Lect. Notes Comput. Sci., 1991, vol. 494.Google Scholar
  2. 2.
    Milner, R., Modal Characterization of Observable Machine Behavior, Proc. CAAP, 1981, Astesiano, G. and Bohm, C. Eds., Lect. Notes Comput. Sci., 1981, vol. 112, pp. 25–34.Google Scholar
  3. 3.
    Van Glabbeek, R.J., The Linear Time—Branching Time Spectrum, Proc. of CONCUR’90, Baeten, J.C.M. and Klop, J.W., Eds., Lect. Notes Comput. Sci., 1990, vol. 458, pp. 278–297.Google Scholar
  4. 4.
    Van Glabbeek, R.J., The Linear Time—Branching Time Spectrum II: The Semantics of Sequential Processes with Silent Moves, Proc. of CONCUR’93, Hildesheim, Germany, 1993, Best, E., Ed., Lect. Notes Comput. Sci., 1993, vol. 715, p. 66.Google Scholar
  5. 5.
    Lynch, N.A. and Tuttle, M.R., Hierarchical Correctness Proofs for Distributed Algorithms, Proc. of the 6th ACM SIGACT-SIGOPS Symp. on Principles of Distributed Computing, 1987, pp. 137–151.Google Scholar
  6. 6.
    Phalippou, M. Relations d’Implantation et Hypotheses de Test des Automates a Entrees et Sorties, PhD Thesis, l’Universite de Bordeaux 1, 1994.Google Scholar
  7. 7.
    Tretmans, J., Test Generation with Inputs, Outputs and Repetitive Quiescence, Software Concepts and Tools, 1996, vol. 17, issue 3.Google Scholar
  8. 8.
    Milner, R., A Calculus of Communicating Systems, Lect. Notes Comput. Sci., 1980, vol. 92.Google Scholar
  9. 9.
    Milner, R., Communication and Concurrency, Prentice-Hall, 1989.Google Scholar
  10. 10.
    Vaandrager, F., On the Relationship between Process Algebra and Input/Output Automata, Logic in Computer Science, Sixth Annual IEEE Symposium, IEEE Computer Society, 1991, pp. 387–398.Google Scholar
  11. 11.
    Jard, C., Jéron, T., Tanguy, L., and Viho, C., Remote Testing Can Be as Powerful as Local Testing, Formal Methods for Protocol Engineering and Distributed Systems, FORTE XII/PSTV XIX’99 Wu, J., Chanson, S., and Gao, Q., Eds., Beijing, 1999, pp. 25–40.Google Scholar
  12. 12.
    Van der Bijl, M., Rensink, A., and Tretmans, J., Compositional Testing with ioco, Formal Approaches to Software Testing: Third Int. Workshop FATES, Petrenko, A. and Ulrich, A., Eds., Montreal, 2003, Lect. Notes Comput. Sci., 2003, vol. 2931, pp. 86–100.Google Scholar
  13. 13.
    Van der Bijl, M., Rensink, A., and Tretmans, J., Component Based Testing with ioco, CTIT Technical Report, Univ. of Twente, 2003, no. TR-CTIT-03-34.Google Scholar
  14. 14.
    Von Bochmann, G.V. and Petrenko, A., Protocol Testing: Review of Methods and Relevance for Software Testing, Proc. of ISSTA, 1994, pp. 109–124.Google Scholar
  15. 15.
    Lee, D. and Yannakakis, M., Principles and Methods of Testing Finite State Machines: A Survey, Proceedings of the IEEE, vol. 84, no. 8, pp. 1090–1123, Berlin: IEEE Computer Society, 1996.Google Scholar
  16. 16.
    Tretmans, J., Formal Approaches to Conformance Testing, Ph.D. Thesis, Enschede, Netherlands: Univ. of Twente, 1992.Google Scholar
  17. 17.
    Bourdonov, I.B. and Kossatchev, A.S., Testing Components of a Distributed System, Trudy Vserossiiskoi konferentsii “Nauchnyi servis v seti Internet” (Proc. of the All-Russia Conf. on the Research Services on the Internet), Moscow: Mosk. Gos. Univ., 2005, pp. 63–65.Google Scholar
  18. 18.
    Bourdonov, I.B. and Kossatchev, A.S., Verification of the Composition of a Distributed System, Trudy Vserossiiskoi konferentsii “Nauchnyi servis v seti Internet” (Proc. of the All-Russia Conf. on the Research Services on the Internet), Moscow: Mosk. Gos. Univ., 2005, pp. 67–69.Google Scholar
  19. 19.
    Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Formal Conformance Testing of Systems with Refused Inputs and Forbidden Actions, Proc. of MBT, Vienna, 2006.Google Scholar
  20. 20.
    Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Teoriya sootvetstviya dlya sistem s blokirovkami i razrusheniyami (Conformance Theory for Systems with Refusals and Destruction), Moscow: Nauka (in press).Google Scholar
  21. 21.
    Heerink, L. and Tretmans, J., Refusal Testing for Classes of Transition Systems with Inputs and Outputs, in Formal Description Techniques and Protocol Specification, Testing and Verification, Chapman & Hill, 1997.Google Scholar
  22. 22.
    Heerink, L. Ins and Outs in Refusal Testing, PhD Thesis, Enschede, Netherlands: Univ. of Twente, 1998.Google Scholar
  23. 23.
    Petrenko, A., Yevtushenko, N., and Huo, J.L., Testing Transition Systems with Input and Output Testers, Proc. 15th Int. Conf. on Communicating Systems, TestCom’2003, Sophia, Antipolis, France, pp. 129–145.Google Scholar
  24. 24.
    Lestiennes, G. and Gaudel, M.-C., Test de Systemes Reactifs non Receptifs, J. Europ. des Systemes Automatises, Modelisation des Systemes Reactifs, 2005, pp. 255–270.Google Scholar
  25. 25.
    Hoare, C.A.R., An Axiomatic Basis for Computer Programming, Commun. ACM, 1969, vol. 12, no. 10, pp. 576–585.MATHCrossRefGoogle Scholar
  26. 26.
    Blass, A., Gurevich, Y., Nachmanson, L., and Veanes, M., Play to Test Microsoft Research, Technical Report, 2005, no. MSR-TR-2005-04; 5th Int. Workshop on Formal Approaches to Testing of Software (FATES 2005), Edinburgh, 2005.Google Scholar
  27. 27.
    Fujiwara, S. and von Bochmann, G. Testing Nondeterministic Finite State Machine with Fault Coverage, Proc. IFIP TC6 Fourth Int. Workshop on Protocol Test Systems, 1991, Kroon, J., Heijink, R.J., and Brinksma E., Eds., North-Holland, 1992, pp. 267–280.Google Scholar
  28. 28.
    Petrenko, A., Yevtushenko, N., and von Bochmann, G., Testing Deterministic Implementations from Nondeterministic FSM Specifications, Selected Proc. of the IFIP TC6 9th Int. Workshop on Testing of Communicating Systems, 1996.Google Scholar
  29. 29.
    Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Application of Finite Automatons for Program Testing, Programmirovanie, 2000, no. 2, pp. 12–28 [Programming Comput. Software (Engl. Transl.), 2000, vol. 26, no. 2, pp. 61–73].Google Scholar
  30. 30.
    Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Irredundant Algorithms for Traversing Directed Graphs: The Deterministic Case, Programmirovanie, 2003, no. 5, pp. 11–30 [Programming Comput. Software (Engl. Transl.), 2003, vol. 29, no. 5, pp. 245–258].Google Scholar
  31. 31.
    Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Irredundant Algorithms for Traversing Directed Graphs: The Nondeterministic Case, Programmirovanie, 2004, no. 1, pp. 4–24 [Programming Comput. Software (Engl. Transl.), 2004, vol. 30, no. 1, pp. 2–17].Google Scholar
  32. 32.
    Kuliamin, V.V., Petrenko, A.K., Kossatchev, A.S., and Bourdonov, I.B., The UniTesK Approach to Designing Test Suites, Programmirovanie, 2003, no. 6, pp. 25–43 [Programming Comput. Software (Engl. Transl.), 2003, vol. 29, no. 6, pp. 310–322].Google Scholar
  33. 33.
    Kuliamin, V.V., Kossatchev, A.S., Petrenko, A.K., Pakoulin, N.V., and Bourdonov, I.B., Integration of Functional and Timed Testing of Real-Time and Concurrent Systems, in Perspectives of System Informatics, Lect. Notes Comput. Sci., 2003.Google Scholar
  34. 34.
    Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Asynchronous Automata: Classification and Testing, Tr. ISP RAN (Proceedings of Institute for System Programming, Russian Academy of Sciences), 2003, vol. 4, pp. 7–84.Google Scholar
  35. 35.
    De Nicola, R. and Hennessy, M.C.B., Testing Equivalence for Processes, Theor. Comput. Sci., 1984, vol. 34, pp. 83–133.MATHCrossRefGoogle Scholar
  36. 36.
    Langerak, R. A., Testing Theory for LOTOS Using Deadlock Detection, in Protocol Specification, Testing, and Verification IX, Brinksma E., Scollo, G., and Vissers, C.A., Eds. North-Holland, 1990, pp. 87–98.Google Scholar

Copyright information

© Pleiades Publishing, Ltd. 2007

Authors and Affiliations

  • I. B. Bourdonov
    • 1
  • A. S. Kossatchev
    • 1
  • V. V. Kuliamin
    • 1
  1. 1.Institute for System ProgrammingRussian Academy of SciencesMoscowRussia

Personalised recommendations