Advertisement

A conformance test suite for TTCN-3 tools

Black-Box functional testing of TTCN-3 syntax and semantics
  • Benjamin ZeissEmail author
  • Andras Kovacs
  • Nikolay Pakulin
  • Bogdan Stanca-Kaposta
TTCN-3

Abstract

With more than 10 years of maturing through industrial use and standardization, the Testing and Test Control Notation (TTCN-3) has become a widely used technology that many businesses depend upon for ensuring their product quality. With the rising number of supporters and tools, the demand for a means to assess the standards compliance of TTCN-3 tools has increased. In this article, we describe the motivation, approach, methodology, and results of the still ongoing project to develop a standardized conformance test suite for TTCN-3 tools. We discuss the challenges involved in creating such a test suite, the way to deal with imposed resource limitations of the project, and where we think the effort is heading.

Keywords

TTCN-3 Syntax tests Semantics tests Conformance testing Compiler testing 

Abbreviations

ATS

Abstract Test Suite

ETS

Executable Test Suite

ETSI

European Telecommunications Standards Institute

LTE

Long-Term Evolution

TTCN-3

Testing and Test Control Notation

IUT

Implementation under Test

STF

Specialist Task Force

IXIT

Implementation eXtra Information for Testing

ICS

Implementation Conformance Statement

TCK

Technology Compatibility Kit

JCK

Java Compatibility Kit

CTT

Compatibility Test Tools

JCP

Java Community Process

JSR

Java Specification Request

RI

Reference Implementation

ISO

International Organization for Standardization

IMS

IP Multimedia Subsystem

ITS

Intelligent Transportation Systems

EBNF

Extended Backus–Naur Form

Notes

Acknowledgments

We would like to thank a lot of people and persons from companies, institutes, and persons who supported this work and who provided valuable feedback (no specific order): ETSI, Conformiq, Jens Grabowski (University of Göttingen), Testing Technologies, Broadbit, Elvior, MTP, ISPRAS, Ericsson, Fraunhofer Fokus, Telecom Italia. We apologize for anyone we have missed to mention.

References

  1. 1.
    Ford, B., et al.: RubySpec. http://rubyspec.org
  2. 2.
    Esin, A., Novikov, A., Yavorskiy, R.: Experiments on semantics based testing of a compiler. http://www.mi.ras.ru/rey/papers/asm05filatov.pdf (2004)
  3. 3.
    ETSI. TTCN-3 Home Page. http://www.ttcn-3.org
  4. 4.
    European Telecommunications Standards Institute (ETSI): ETSI ES 201 873 V4.2.1 (2010–07): Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Parts 1–10 (2010)Google Scholar
  5. 5.
    European Telecommunications Standards Institute (ETSI): ETSI ES 102 950 V1.1.1 (2011–04): Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Parts 1–3 (2011)Google Scholar
  6. 6.
    European Telecommunications Standards Institute (ETSI): ETSI ES 102 950 V1.2.1 (2012–04): Methods for Testing and Specification (MTS); TTCN-3 Conformance Test Suite; Parts 1–3 (2012)Google Scholar
  7. 7.
    ISO. ISO/IEC 9899:1999—Programming Languages - C (1999)Google Scholar
  8. 8.
    ISO. ISO/IEC 14882:2011—Information technology—Program- ming languages-C++ (2011)Google Scholar
  9. 9.
    Java Community Process: Java Compatibility Test Tools (Java CTT). http://jcp.org/en/resources/tdk
  10. 10.
    Java Community Process: JSR 901: Java Language Specification. http://jcp.org/en/jsr/detail?id=901
  11. 11.
    Xavier, L.: Formal certification of a compiler back-end or: programming a compiler with a proof assistant. SIGPLAN Not. 41, 42–54 (2006) Google Scholar
  12. 12.
    Matsumoto, Y.: Ruby. http://www.ruby-lang.org
  13. 13.
    Perennial: C++VS: Perennial C++ Validation Suite/CSVA. http://www.peren.com
  14. 14.
    Hall, P.: The Plum Hall Validation Suite for C/Suite++/ LibSuite++. http://www.plumhall.com/
  15. 15.
    Zelenova, S.A., Zelenov, S.V.: Model-based testing of optimizing compilers. In Testing of Software and Communicating Systems, volume 4581 of Lecture Notes in Computer Science (LNCS), pages 365–377 (2007)Google Scholar
  16. 16.
    Thomas, D., Hunt, A.: RubyTests/Rubicon. http://rubyforge.org/projects/rubytests
  17. 17.
    van Rossum, G.: Python. http://www.python.org

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Benjamin Zeiss
    • 1
    Email author
  • Andras Kovacs
    • 2
  • Nikolay Pakulin
    • 3
  • Bogdan Stanca-Kaposta
    • 4
  1. 1.T-Systems International GmbHDarmstadtGermany
  2. 2.BroadBitKomarnoSlovakia
  3. 3.Institute for System Programming of the Russian Academy of Sciences (ISPRAS)MoscowRussia
  4. 4.Testing Technologies IST GmbHBerlinGermany

Personalised recommendations