A conformance test suite for TTCN-3 tools

Black-Box functional testing of TTCN-3 syntax and semantics
  • Benjamin ZeissEmail author
  • Andras Kovacs
  • Nikolay Pakulin
  • Bogdan Stanca-Kaposta


With more than 10 years of maturing through industrial use and standardization, the Testing and Test Control Notation (TTCN-3) has become a widely used technology that many businesses depend upon for ensuring their product quality. With the rising number of supporters and tools, the demand for a means to assess the standards compliance of TTCN-3 tools has increased. In this article, we describe the motivation, approach, methodology, and results of the still ongoing project to develop a standardized conformance test suite for TTCN-3 tools. We discuss the challenges involved in creating such a test suite, the way to deal with imposed resource limitations of the project, and where we think the effort is heading.


TTCN-3 Syntax tests Semantics tests Conformance testing Compiler testing 



Abstract Test Suite


Executable Test Suite


European Telecommunications Standards Institute


Long-Term Evolution


Testing and Test Control Notation


Implementation under Test


Specialist Task Force


Implementation eXtra Information for Testing


Implementation Conformance Statement


Technology Compatibility Kit


Java Compatibility Kit


Compatibility Test Tools


Java Community Process


Java Specification Request


Reference Implementation


International Organization for Standardization


IP Multimedia Subsystem


Intelligent Transportation Systems


Extended Backus–Naur Form



We would like to thank a lot of people and persons from companies, institutes, and persons who supported this work and who provided valuable feedback (no specific order): ETSI, Conformiq, Jens Grabowski (University of Göttingen), Testing Technologies, Broadbit, Elvior, MTP, ISPRAS, Ericsson, Fraunhofer Fokus, Telecom Italia. We apologize for anyone we have missed to mention.


  1. 1.
    Ford, B., et al.: RubySpec.
  2. 2.
    Esin, A., Novikov, A., Yavorskiy, R.: Experiments on semantics based testing of a compiler. (2004)
  3. 3.
    ETSI. TTCN-3 Home Page.
  4. 4.
    European Telecommunications Standards Institute (ETSI): ETSI ES 201 873 V4.2.1 (2010–07): Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Parts 1–10 (2010)Google Scholar
  5. 5.
    European Telecommunications Standards Institute (ETSI): ETSI ES 102 950 V1.1.1 (2011–04): Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Parts 1–3 (2011)Google Scholar
  6. 6.
    European Telecommunications Standards Institute (ETSI): ETSI ES 102 950 V1.2.1 (2012–04): Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Parts 1–3 (2012)Google Scholar
  7. 7.
    ISO. ISO/IEC 9899:1999—Programming Languages - C (1999)Google Scholar
  8. 8.
    ISO. ISO/IEC 14882:2011—Information technology—Program- ming languages-C++ (2011)Google Scholar
  9. 9.
    Java Community Process: Java Compatibility Test Tools (Java CTT).
  10. 10.
    Java Community Process: JSR 901: Java Language Specification.
  11. 11.
    Xavier, L.: Formal certification of a compiler back-end or: programming a compiler with a proof assistant. SIGPLAN Not. 41, 42–54 (2006) Google Scholar
  12. 12.
    Matsumoto, Y.: Ruby.
  13. 13.
    Perennial: C++VS: Perennial C++ Validation Suite/CSVA.
  14. 14.
    Hall, P.: The Plum Hall Validation Suite for C/Suite++/ LibSuite++.
  15. 15.
    Zelenova, S.A., Zelenov, S.V.: Model-based testing of optimizing compilers. In Testing of Software and Communicating Systems, volume 4581 of Lecture Notes in Computer Science (LNCS), pages 365–377 (2007)Google Scholar
  16. 16.
    Thomas, D., Hunt, A.: RubyTests/Rubicon.
  17. 17.
    van Rossum, G.: Python.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Benjamin Zeiss
    • 1
    Email author
  • Andras Kovacs
    • 2
  • Nikolay Pakulin
    • 3
  • Bogdan Stanca-Kaposta
    • 4
  1. 1.T-Systems International GmbHDarmstadtGermany
  2. 2.BroadBitKomarnoSlovakia
  3. 3.Institute for System Programming of the Russian Academy of Sciences (ISPRAS)MoscowRussia
  4. 4.Testing Technologies IST GmbHBerlinGermany

Personalised recommendations