Skip to main content

Analyzing Response Inconsistencies in Test Suites

  • Conference paper
  • 636 Accesses

Part of the Lecture Notes in Computer Science book series (LNCCN,volume 5826)


Extensive testing of modern communicating systems often involve large and complex test suites that need to be maintained throughout the life cycle of the tested system. For this purpose, quality assurance of test suites is an inevitable task that eventually has an impact on the quality of the system under test as well. In this work, we present a means to analyze response inconsistencies in test suites. We define a response consistency relation and describe a method that identifies candidates for the analysis. Using these candidates, we find response inconsistent states. The applicability of this method is discussed for local test cases, local test cases with different response orders, and distributed test cases with concurrent behavior.


  • Test Suite
  • Session Initiation Protocol
  • Stimulus Sequence
  • Label Transition System
  • System Under Test

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Parnas, D.: Software Aging. In: Proceedings of the 16th International Conference on Software Engineering (ICSE), Sorrento, Italy, May 16–21, pp. 279–287. IEEE Computer Society/ACM Press (1994)

    Google Scholar 

  2. Neukirchen, H., Zeiss, B., Grabowski, J., Baker, P., Evans, D.: Quality Assurance for TTCN-3 Test Specifications. Software Testing, Verification and Reliability (STVR) 18(2) (2008)

    Google Scholar 

  3. Neukirchen, H., Zeiss, B., Grabowski, J.: An Approach to Quality Engineering of TTCN-3 Test Specifications. International Journal on Software Tools for Technology Transfer (STTT) 10(4), 309–326 (2008)

    CrossRef  Google Scholar 

  4. TRex Website, (Last Checked May 15, 2009)

  5. European Telecommunications Standards Institute (ETSI): ETSI ES 201 873 V3.4.1 (2008-2009): The Testing and Test Control Notation version 3; Parts 1–10 (2008)

    Google Scholar 

  6. Zeiss, B., Grabowski, J.: Reverse-Engineering Test Behavior Models for the Analysis of Structural Anomalies. In: Suzuki, K., Higashino, T., Ulrich, A., Hasegawa, T. (eds.) TestCom/FATES 2008. LNCS, vol. 5047. Springer, Heidelberg (2008)

    Google Scholar 

  7. Din, G., Vega, D., Schieferdecker, I.: Automated Maintainability of TTCN-3 Test Suites Based on Guideline Checking. In: Brinkschulte, U., Givargis, T., Russo, S. (eds.) SEUS 2008. LNCS, vol. 5287, pp. 417–430. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  8. Vega, D., Din, G., Taranu, S., Schieferdecker, I.: Application of Clustering Methods for Analysing of TTCN-3 Test Data Quality. In: Proceedings of the 2008 The Third International Conference on Software Engineering Advances, ICSEA 2008 (2008)

    Google Scholar 

  9. Vega, D., Schieferdecker, I., Din, G.: Test Data Variance as a Test Quality Measure: Exemplified for TTCN-3. In: Petrenko, A., Veanes, M., Tretmans, J., Grieskamp, W. (eds.) TestCom/FATES 2007. LNCS, vol. 4581, pp. 351–364. Springer, Heidelberg (2007)

    CrossRef  Google Scholar 

  10. Zeiss, B., Vega, D., Schieferdecker, I., Neukirchen, H., Grabowski, J.: Applying the ISO 9126 Quality Model to Test Specifications – Exemplified for TTCN-3 Test Specifications. In: Proceedings of Software Engineering 2007 (SE 2007). Lecture Notes in Informatics (LNI), vol. 105. Köllen Verlag (2007)

    Google Scholar 

  11. Boroday, S., Petrenko, A., Ulrich, A.: Test Suite Consistency Verification. In: Proceedings of the 6th IEEE East-West Design & Test Symposium (EWDTS 2008), Ukraine (2008)

    Google Scholar 

  12. Cartaxo, E.G., Neto, F.G.O., Machado, P.D.L.: Automated Test Case Selection Based on a Similarity Function. In: Proceedings of the 2nd Workshop on Model-Based Testing (MOTES 2007) (2007)

    Google Scholar 

  13. Alilovic-Curgus, J., Vuong, S.T.: A Metric Based Theory of Test Selection and Coverage. In: Proceedings of the IFIP TC6/WG6.1 Thirteenth International Symposium on Protocol Specification, Testing and Verification XIII, North-Holland, pp. 289–304 (1993)

    Google Scholar 

  14. Utting, M., Legeard, B.: Practical Model-Based Testing – A Tools Approach. Morgan Kaufmann Publishers, San Francisco (2007)

    Google Scholar 

  15. Tretmans, J.: Test Generation with Inputs, Outputs, and Quiescence. In: Margaria, T., Steffen, B. (eds.) TACAS 1996. LNCS, vol. 1055. Springer, Heidelberg (1996)

    CrossRef  Google Scholar 

  16. Milner, R.: A Calculus of Communication Systems. LNCS, vol. 92. Springer, Heidelberg (1980)

    CrossRef  MATH  Google Scholar 

  17. ETSI: TS 102 027-3: SIP ATS & PIXIT; Part 3: Abstract Test Suite (ATS) and partial Protocol Implementation eXtra Information for Testing (PIXIT). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zeiss, B., Grabowski, J. (2009). Analyzing Response Inconsistencies in Test Suites. In: Núñez, M., Baker, P., Merayo, M.G. (eds) Testing of Software and Communication Systems. FATES TestCom 2009 2009. Lecture Notes in Computer Science, vol 5826. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05030-5

  • Online ISBN: 978-3-642-05031-2

  • eBook Packages: Computer ScienceComputer Science (R0)