Integrated System Interoperability Testing with Applications to VoIP
This work has been motivated by the need to test interoperability of systems carrying voice calls over the IP network. Voice over IP (VoIP) systems must be integrated and interoperate with the existing Public Switched Telephone Network (PSTN) before they will be widely adopted. Standards have been developed to address this problem, but unfortunately different standards bodies and commercial consortiums have defined different standards. Furthermore, the prevailing commercial standard (H.323) is incomplete, complex, and presents the implementers with “vendors latitudes”. As a result, there is no guarantee that the integrated VoIP systems would interoperate properly even if the implementations are all H.323-compliant. Interoperability testing has become indispensable.
We discuss criteria that cover all the required patterns of “inter-operating behavior.” We want to test all such system interoperations. On the other hand, test execution in real environment is expensive, and we want to minimize the number of tests while maintaining the coverage.
We present a general method for automatic generation of test cases, which cover all the required system interoperations and contain a minimal number of tests. We study data structures and efficient test generation algorithms, which take time proportional to the total test case size. We report experimental results on VoIP systems.
KeywordsVoIP H.323 protocol integrated system interoperability testing coverage redundancy
- LATA switching systems generic requirements: Call processing. Technical Report GR-505-CORE, Bellcore, December 1997.Google Scholar
- A.V. Aho, J. E. Hoperoft, and J. D. Ullman. The Design and Analysis of Computer Algorithms. Addison-Wesley, 1974.Google Scholar
- G. Bonnes. IBM OSI interoperability verification services. In IFIP TC6 WG6.1 The 3rd International Workshop on Protocol Test System, 1990.Google Scholar
- R. Castanet and O. Kone. Deriving coordinated testers for inter-operability. In O. Rafiq, editor, Protocol Test Systems, VI(C-19), pages 331–345. Elsevier Science Publisher B. V.(North-Holland), 1994.Google Scholar
- J. Gadre, Rohre C, C. Summers, and S. Symington. A COS study of OSI interoperability. Computer Standards and Interfaces, 9(3):217237, 1990.Google Scholar
- R. Hao. Protocol Conformance and Interoperability Testing based on Formal Methods. PhD thesis, Tsinghua University, P.R.China, 1997.Google Scholar
- S. Kang and M. Kim. Test sequence generation for adaptive inter-operability testing. In Proceeding of Protocol Test Systems, VIII, pages 187–200, 1995.Google Scholar
- S. Kang and M. Kim. Interoperability test suite deriva- tion for symmetric communication protocols. In Proceeding of FORTE/PSTV’97, 1997.Google Scholar
- D. Lee and M. Yannakakis. Principles and methods of testing finite state machines - a survey. The Proceedings of IEEE, 84(8):10891123, August 1996.Google Scholar
- N.Arakawa and T. Soneoka. A test case generation method for concurrent programs. In R.J. Heijink J. Kroon and E. Brinksma, editors, Protocol Test Systems, IV, pages 95–106. Elsevier Science Publisher B. V.(North-Holland), 1992.Google Scholar
- O Rafiq and R. Castanet. From conformance testing to interoperability testing. In The 3rd International Workshop on Protocol Test Systems, 1990.Google Scholar
- G. S. Vermeer and H. Blik. Interoperability testing: Basis for the acceptance of communicating systems. In Protocol Test Systems, VI(C-19). Elsevier Science Publisher B. V.(North-Holland), 1994.Google Scholar