Skip to main content
SpringerLink
Log in

Timed implementation relations for the distributed test architecture

  • Published:
Distributed Computing Aims and scope Submit manuscript

Abstract

In order to test systems that have physically distributed interfaces, called ports, we might use a distributed approach in which there is a separate tester at each port. If the testers do not synchronise during testing then we cannot always determine the relative order of events observed at different ports and this leads to new notions of correctness that have been described using corresponding implementation relations. We study the situation in which each tester has a local clock and timestamps its observations. If we know nothing about how the local clocks relate then this does not affect the implementation relation while if the local clocks agree exactly then we can reconstruct the sequence of observations made. In practice, however, we are likely to be between these extremes: the local clocks will not agree exactly but we have some information regarding how they can differ. We start by assuming that a local tester interacts synchronously with the corresponding port of the system under test and then extend this to the case where communications can be asynchronous, considering both the first-in-first-out (FIFO) case and the non-FIFO case. The new implementation relations are stronger than implementation relations for distributed testing that do not use timestamps but still reflect the distributed nature of observations. This paper explores these alternatives and derives corresponding implementation relations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. If an input cannot be applied in some state of the SUT, then we can assume that there is a response to the input that reports that this input is blocked.

  2. It is possible to remove this restriction by considering infinite traces rather than quiescent traces [21] but this complicates the exposition.

  3. In the graphical representation we have omitted irrelevant input transitions. For each state \(q\) and input \(?i\in \{?open_1,?open_2,?save_1,?save_2\}\), if there does not exist an outgoing transition from \(q\) labelled by \(?i\) then we assume the existence of the transition \((q,?i,q)\).

  4. We assume that an output cannot be delayed past quiescence but it is straightforward to change the definition in order to remove this assumption.

  5. Again we assume that an output cannot be delayed past quiescence but it is straightforward to change the definition in order to remove this assumption.

  6. The membership problem is that of determining whether a given trace is a trace of the specification.

References

  1. Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, Cambridge (2008)

    Book  MATH  Google Scholar 

  2. Bhateja, P., Gastin, P., Mukund, M.: A fresh look at testing for asynchronous communication. In: 4th International Symposium on Automated Technology for Verification and Analysis, ATVA’06, LNCS 4218, pp. 369–383. Springer (2006)

  3. Boreale, M., De Nicola, R., Pugliese, R.: Asynchronous observations of processes. In: 1st International Conference on the Foundations of Software Science and Computation Structure, FoSSaCS’98, LNCS 1378, pp. 95–109. Springer (1998)

  4. Boreale, M., De Nicola, R., Pugliese, R.: A theory of “may” testing for asynchronous languages. In: 2nd International Conference on the Foundations of Software Science and Computation Structure, FoSSaCS’99, LNCS 1578, pp. 165–179. Springer (1999)

  5. Boyd, S., Ural, H.: The synchronization problem in protocol testing and its complexity. Inf. Process. Lett. 40(3), 131–136 (1991)

    Article  MATH  MathSciNet  Google Scholar 

  6. Cacciari, L., Rafiq, O.: Controllability and observability in distributed testing. Inf. Softw. Technol. 41(11–12), 767–780 (1999)

    Article  Google Scholar 

  7. Castellani, I., Hennessy, M.: Testing theories for asynchronous languages. In: 18th Conference on Foundations of Software Technology and Theoretical Computer Science, FSTTCS’98, LNCS 1530, pp. 90–101. Springer (1998)

  8. Chen, W., Ural, H.: Synchronizable checking sequences based on multiple UIO sequences. IEEE/ACM Trans. Netw. 3, 152–157 (1995)

    Article  Google Scholar 

  9. Cunha de Almeida, E., Marynowski, J., Sunyé, G., Traon, Y.L., Valduriez, P.: Efficient distributed test architecture for large-scale systems. In: 22nd International Conference on Testing Software and Systems, ICTSS’10, LNCS 6435, pp. 174–187. Springer (2010)

  10. Dssouli, R., von Bochmann, G.: Conformance testing with multiple observers. In: 6th WG6.1 International Conference on Protocol Specification, Testing and Verification, PSTV’86, pp. 217–229. North-Holland (1986)

  11. Farchi, E., Hartman, A., Pinter, S.: Using a model-based test generator to test for standard conformance. IBM Syst. J. 41(1), 89–110 (2002)

    Article  Google Scholar 

  12. Fidge, C.: A limitation of vector timestamps for reconstructing distributed computations. Inf. Process. Lett. 68(2), 87–91 (1998)

    Article  Google Scholar 

  13. Gaudel, M.C.: Testing can be formal, too! In: 6th International Joint Conference CAAP/FASE, Theory and Practice of Software Development, TAPSOFT’95, LNCS 915, pp. 82–96. Springer (1995)

  14. Gonenc, G.: A method for the design of fault detection experiments. IEEE Trans. Comput. 19, 551–558 (1970)

    Article  Google Scholar 

  15. Grieskamp, W., Kicillof, N., Stobie, K., Braberman, V.: Model-based quality assurance of protocol documentation: tools and methodology. Softw. Test. Verif. Reliab. 21(1), 55–71 (2011)

    Article  Google Scholar 

  16. Hierons, R.M.: Oracles for distributed testing. IEEE Trans. Softw. Eng. 38(3), 629–641 (2012)

    Google Scholar 

  17. Hierons, R.M., Bogdanov, K., Bowen, J., Cleaveland, R., Derrick, J., Dick, J., Gheorghe, M., Harman, M., Kapoor, K., Krause, P., Luettgen, G., Simons, A., Vilkomir, S., Woodward, M., Zedan, H.: Using formal specifications to support testing. ACM Comput. Surv. 41(2), 1–76 (2009)

    Google Scholar 

  18. Hierons, R.M., Bowen, J., Harman, M. (eds.): Formal Methods and Testing, LNCS 4949. Springer, Berlin (2008)

    Google Scholar 

  19. Hierons, R.M., Merayo, M.G., Núñez, M.: Controllable test cases for the distributed test architecture. In: 6th International Symposium on Automated Technology for Verification and Analysis, ATVA’08, LNCS 5311, pp. 201–215. Springer (2008)

  20. Hierons, R.M., Merayo, M.G., Núñez, M.: Implementation relations for the distributed test architecture. In: Joint 20th IFIP TC6/WG6.1 Int. Conf. on Testing of Software and Communicating Systems, TestCom’08, and 8th International Workshop on Formal Approaches to Software Testing, FATES’08, LNCS 5047, pp. 200–215. Springer (2008)

  21. Hierons, R.M., Merayo, M.G., Núñez, M.: Implementation relations and test generation for systems with distributed interfaces. Distrib. Comput. 25(1), 35–62 (2012)

    Article  MATH  Google Scholar 

  22. Hierons, R.M., Merayo, M.G., Núñez, M.: Using time to add order to distributed testing. In: 18th Symposium on Formal Methods, FM’12, LNCS 7436, pp. 232–246. Springer (2012)

  23. Hierons, R.M., Ural, H.: Checking sequences for distributed test architectures. Distrib. Comput. 21(3), 223–238 (2008)

    Article  MATH  Google Scholar 

  24. ISO/IEC JTC 1, J.T.C.: International Standard ISO/IEC 9646–1. Information Technology—Open Systems Interconnection—Conformance testing methodology and framework-Part 1: General concepts. ISO/IEC (1994)

  25. Jard, C., Jéron, T., Kahlouche, H., Viho, C.: Towards automatic distribution of testers for distributed conformance testing. In: TC6 WG6.1 Joint International Conference on Formal Description Techniques and Protocol Specification, Testing and Verification, FORTE’98, pp. 353–368. Kluwer Academic Publishers (1998)

  26. Jard, C., Jéron, T., Tanguy, L., Viho, C.: Remote testing can be as powerful as local testing. In: 19th Joint International Conference on Protocol Specification, Testing, and Verification and Formal Description Techniques, FORTE/PSTV’99, pp. 25–40. Kluwer Academic Publishers (1999)

  27. Kim, M., Shin, J., Chanson, S.T., Kang, S.: An enhanced model for testing asynchronous communicating systems. In: 19th Joint International Conference on Protocol Specification, Testing, and Verification and Formal Description Techniques, FORTE/PSTV’99, pp. 337–356. Kluwer Academic Publishers (1999)

  28. Lamport, L.: Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21(7), 558–565 (1978)

    Article  MATH  Google Scholar 

  29. Lamport, L., Melliar-Smith, P.: Synchronizing clocks in the presence of faults. J. ACM 32(1), 52–78 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  30. Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines: a survey. Proc. IEEE 84(8), 1090–1123 (1996)

    Article  Google Scholar 

  31. Luo, G., Dssouli, R., von Bochmann, G.: Generating synchronizable test sequences based on finite state machine with distributed ports. In: 6th IFIP Workshop on Protocol Test Systems, IWPTS’93, pp. 139–153. North-Holland (1993)

  32. Moore, E.: Gedanken experiments on sequential machines. In: Shannon, C., McCarthy, J. (eds.) Automata Studies. Princeton University Press, Princeton, NJ (1956)

    Google Scholar 

  33. Myers, G.: The Art of Software Testing, 2nd edn. Wiley, London (2004)

    Google Scholar 

  34. Ostrovsky, R., Patt-Shamir, B.: Optimal and efficient clock synchronization under drifting clocks. In: 18th Annual ACM Symposium on Principles of Distributed Computing, PODC’99, pp. 3–12. ACM Press (1999)

  35. Petrenko, A., Yevtushenko, N.: Testing from partial deterministic FSM specifications. IEEE Trans. Comput. 54(9), 1154–1165 (2005)

    Article  Google Scholar 

  36. Rafiq, O., Cacciari, L.: Coordination algorithm for distributed testing. J. Supercomput. 24(2), 203–211 (2003)

    Article  MATH  Google Scholar 

  37. Sarikaya, B., von Bochmann, G.: Synchronization and specification issues in protocol testing. IEEE Trans. Commun. 32, 389–395 (1984)

    Google Scholar 

  38. Tai, K.C., Young, Y.C.: Synchronizable test sequences of finite state machines. Comput. Netw. ISDN Syst. 30(12), 1111–1134 (1998)

    Article  Google Scholar 

  39. Tretmans, J.: Conformance testing with labelled transition systems: implementation relations and test generation. Comput. Netw. ISDN Syst. 29, 49–79 (1996)

    Google Scholar 

  40. Tretmans, J.: Model based testing with labelled transition systems. In: Hierons R.M., Bowen J.P., Harman M. (eds.) Formal Methods and Testing, LNCS 4949, pp. 1–38. Springer, Berlin (2008)

  41. Ural, H., Williams, C.: Constructing checking sequences for distributed testing. Form. Asp. Comput. 18(1), 84–101 (2006)

    Article  MATH  Google Scholar 

  42. Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann, Los Altos, CA (2007)

    Google Scholar 

Download references

Acknowledgments

We would like to thank the anonymous reviewers for the careful reading and for their constructive comments that have helped to further strengthen the paper.

Author information

Authors and Affiliations

  1. Department of Information Systems and Computing, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK

    Robert M. Hierons

  2. Departamento de Sistemas Informáticos y Computación, Universidad Complutense de Madrid, Madrid, Spain

    Mercedes G. Merayo & Manuel Núñez

Authors
  1. Robert M. Hierons
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mercedes G. Merayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Núñez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mercedes G. Merayo.

Additional information

Research partially supported by the Spanish MICINN project TESIS (TIN2009-14312-C02-01) and the UK EPSRC project Testing of Probabilistic and Stochastic Systems (EP/G032572/1). This research was carried out while the second author was visiting Brunel University supported by a grant Fundación Caja Madrid para movilidad de profesores de universidades públicas de Madrid.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hierons, R.M., Merayo, M.G. & Núñez, M. Timed implementation relations for the distributed test architecture. Distrib. Comput. 27, 181–201 (2014). https://doi.org/10.1007/s00446-014-0208-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00446-014-0208-5

Keywords

Search

Navigation