Skip to main content
SpringerLink
Log in

Conformance testing in systems with semicontrollable interfaces

Esssais de conformitÉ dans des systÈmes À interfaces semi-commandables

  • Published:
Annales Des Télécommunications Aims and scope Submit manuscript

Abstract

In a conformance testing environment, an implementation under test (iut) communicates with multiple entities. A tester may have differing degrees of control on the interactions between these entities and the iut : directly controllable, semicontrollable, or uncontrollable. Semi-controllable inputs most likely render portions of an IUT untestable. In addition, multiple communicating entities may create race conditions during testing. This paper presents a test generation methodology for systems where the semicontrollable inputs can be generated indirectly. The test sequences obtained from the converted graph fully utilize the semicontrollable inputs (where possible) while avoiding the race conditions. Although, for the most general case, the graph conversion results in a exponentially large number of nodes, practical considerations make the converted graph size feasible. This approach is used to generate tests for mil-std 188-220B. By applying the proposed graph conversion and the race condition elimination techniques, the number of testable state transitions increased from approximately 200 to over 700, which represents a coverage of 95% of the transitions defined in the specification.

Résumé

Dans un environnement d’essai de conformité, un composant en essai communique avec de multiples entités. Un testeur peut avoir différents degrés de commande sur les interactions entre ces entités et le composant en essai : elles peuvent être directement commandables, semi-com-mandables ou non commandables. Des entrées semi-com-mandables rendent vraisemblablement non-testables certaines parties du composant en essai. En outre, de multiples entités communicantes peuvent créer des états de concurrence pendant les essais. L’article présente une méthodologie de génération de tests pour les systèmes où des entrées semi-commandables peuvent se produire indirectement. Les séquences d’essai obtenues à partir du graphe converti utilisent pleinement les entrées semi-commandables (si possible) tout en évitant les états de concurrence. Bien que, dans le cas le plus général, la conversion du graphe aboutit à un nombre exponentiellement grand de nœuds, des considérations pratiques rendent réalisable la taille du graphe converti. Cette approche est appliquée à la génération de tests pour la norme mil-std 188-220B. En appliquant la méthode proposée de conversion de graphe et les techniques d’élimination des états de concurrence, le nombre de transitions d’état testables croît d’environ 200 à plus de 700, soit une couverture à 95% des transitions définies dans la spécification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aho (A.V.), Dahbura (A.T.), Lee (D.), Uyar (M.U.). An optimization technique for protocol conformance test generation based on UIO sequences and rural Chinese postman tours,ieee Trans. Commun. 39, n° 11, pp. 1604–1615, (November 1991).

    Article  Google Scholar 

  2. Budkowski (S.), Cavalli (A.), Najm (E.). Eds.Proc IFIP Joint Int’l Conf. FORTE/PSTV (Paris, France, Nov. 1998), Boston, MA:Kluwer Academic Publishers.

    Google Scholar 

  3. Budkowski (S.), Dembinski (P.). An introduction to Estelle: A specification language for distributed systems.Comput. Networks ISDN Syst. 14, n° 1, pp. 3–24. (1991)

    Article  Google Scholar 

  4. Cavaixi (A.R.), Favreau (J.P.), Phallippou (M). Standardization of formal methods in conformance testing of communication protocols.Comput. Networks ISDN Syst. 29, n° l, pp.3–24. (1996)

    Google Scholar 

  5. Cormen (T.H.), Leiserson (C.E.), Rivest (EX.).Introduction to Algorithms.McGraw-Hill, New York, NY. (1992)

    Google Scholar 

  6. *** DoD.Military standard - interoperability standard for Digital Message Device subsystems (mil-std 188-220B). (January 1998)

  7. Fecko (M.A.),Amer (P.D.),Sethi (A.S.),Uyar (M.U.),Dzik (T.),Menell (R.), andMcMahon (M.). Formal design and testing ofmil-std 188-2220B based on Estelle. InProc. ieee Milit. Commun. Conf. (milcom) (Monterey, CA, November 1997)

  8. Fecko (M. A.), Uyar (M. U.), Amer (P. D.), Sethi (A. S.). Using semicontrollable interfaces in testing Army communications protocols: application to mil-std 188-220B. InProc. IEEE Milit. Commun. Conf. (Milcom) (Atlantic City, NJ, Oct. 1999).

    Google Scholar 

  9. Fecko (M. A.), Uyar (M.U.), Sethi (A. S.), Amer (P.D.). Embedded testing in systems with semicontrollable interfaces. Tech. Rep. TR-98-18, CIS Dept., Univ. of Delaware, Newark, DE, (1998).

    Google Scholar 

  10. Fecko (M. A.),Uyar (M. U.),Sethi (A. S.),Amer (P. D.). Issues in conformance testing : Multiple semicontrollable interfaces. In Budkowski et al. [2], pp. 111–126.

  11. Hopcroft (J. E.),Ullman (J. D.).Introduction to automata theory, languages and computation. Addison-Wesley, 1979.

  12. ISO, Information technology- OSI.Iso International standard 9646: Conformance Testing Methodology and Framework. Geneva, Switzerland, 1991.

  13. ***iso/iec. International Standard iso/iec 8802-2, ANSI/IEEE Std. 802.2, ed., (Dec. 1994).

  14. ***Framework: Formal methods in conformance testing, JTCl/SC21/WGl/Project 54. 1. (Feb. 1995).

  15. Kim (M.), Kang (S.), Hong, (K.). Eds.Proc. IFIP Int’l Workshop Test. Communicat. Syst. (IWTCS) (Cheju Island, Korea, Sept. 1997), Boston, MA:Kluwer Academic Publishers.

    Google Scholar 

  16. Lee (D. Y.), Lee (J. Y.). A well-defined Estelle specification for the automatic test generation.IEEE Trans. Comput. 40, n° 4 (Apr. 1991).

    Google Scholar 

  17. Lenstra (J. K.) Rinnooy Kan (A. H. G.). On general routing problems.Networks 6, pp. 273–280 (1976).

    Article  MATH  MathSciNet  Google Scholar 

  18. Lima Jr., (L. P.),Cavalli (A. R.). A pragmatic approach to generating test sequence for embedded systems. InKim et al. [15].

  19. Linn (R. J.).Conformance testing for osi protocols. Comput. Networks ISDN Syst. 18, n°3, pp. 203–219 (1990).

    Article  Google Scholar 

  20. Linn (R. J.), Uyar (M. U.). Conformance testing methodologies and architectures for iso protocols.IEEE Comput. Soc. Press, Los Alamitos, CA, 1994.

    Google Scholar 

  21. Miller (R. E.), Paul (S.). Structural analysis of protocol specifications and generation of maximal fault coverage conformance test sequences.IEEE/ACM Trans. Network. 2, n° 5, pp. 457–470 (Oct. 1994).

    Article  Google Scholar 

  22. Petrenko (A.). Eds.Proc. IFIP Int’l Workshop Test. Communicat. Syst. (IWTCS) (Tomsk, Russia, Sept. 1998). Boston, MA:Kluwer Academic Publishers.

    Google Scholar 

  23. Petrenko (A. F.),Yevtushenko (N). Fault detection in embedded components.In Kim et al. [15].

  24. Petrenko (A. F.),Yevtushenko (N.),Von Bochmann (G.). Fault models for testing in context. InProc.IFIP Joint Int’l Conf. Forte/PSTV, (Kaiserslautern, Germany, Oct. 1996).

  25. Petrenko (A. F.), Yevtushenko (N.), Testing in context: Framework and test derivation.Comput. Commun. 19, n° 14, pp. 1236–1249 (1996).

    Article  Google Scholar 

  26. Phalippou (M.). The limited power of testing. InProc. IFIP Int’l Workshop Protocol Test Syst. (IWPTS), Amsterdam : North-Holland, (1992).

  27. Rafiq (O.),Castanet (R.). From conformance testing to interoperability testing. InProc. IFIP Int’l Workshop Protocol Test Syst. (IWPTS), pp. 371–385 Washington, DC, (1990).

  28. Rayner (D.). Osi conformance testing.Comput. Networks ISDN Syst. 14, n°l, pp. 79–98 (1987).

    Article  Google Scholar 

  29. Sabnani (K. K.),Dahbura (A.T.). A protocol test generation procedure.Comput. Networks ISDN Syst. IS, pp. 285-297 (1988).

  30. Sabnani (K. K.) Uyar (M. U.) Lapone (A. M.). An algorithmic procedure for checking safety properties of communication protocols.IEEE Trans. Commun. 37, n° 9, pp. 940–948 (Sept. 1989).

    Article  Google Scholar 

  31. Timohovich (E.). An approach to protocol entity model development for embedded testing.Automatic Control Comput. Sci. 27, n° 3, pp. 34–41 (1993).

    Google Scholar 

  32. Tretmans (J.). Conformance testing with labelled transitions systems: Implementation relations and test generation.Comput. Networks ISDN Syst. 29, n° 1, pp. 49–79 (1996).

    Article  Google Scholar 

  33. Ural (H.). Formal methods for test sequence generation.Comput. Commun. 15, n° 5, pp. 311–325 (June 1992).

    Article  Google Scholar 

  34. Wu (W.-J), Chen (W.-H.), Tang, (C. Y). Synchronizable test sequence for multi-party protocol conformance testing.Comput. Commun. 21, pp. 1177–1183 (1998).

    Article  Google Scholar 

  35. Yevtushenko (N.), Cavalli (A. R.), Lima JR, (L. P.). Test suite minimization for testing in context. InPetrenko andYevtushenko [22], pp. 127 1945.

    Google Scholar 

  36. Zeng (H. X), Chanson (S. T.), Smith (B. R.). On ferry clip approaches in protocol testing.Comput. Networks isdn Syst. 17 n° 2 (1989) pp 77–88.

    Article  Google Scholar 

  37. Zhu (J.,) Vuong (S. T.), Chanson (S. T.). Evaluation of test coverage for embedded system testing. InPetrenko andYevtushenko [22], pp. 111–126, 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer and information Sciences Department University of Delaware, Newark, DE

    Mariusz A. Fecko, Paul D. Amer & Adarshpal S. Sethi

  2. Electrical Engineering Departement City College of the City University of New York, NY

    M.Ümit Uyar

Authors
  1. Mariusz A. Fecko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M.Ümit Uyar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul D. Amer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adarshpal S. Sethi
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This work is supported by the ARO SSP administered by Batteile (DAAL03-91-C-0034), by ARO (DAAL03-91-G-0086), and by ATIRP Consortium sponsored by the ARL under the FedLab Program (DAAL01-96-2-0002). † Dr. Uyar initiated this research while a Visting Associate Professor at University of Delaware.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fecko, M.A., Uyar, M., Amer, P.D. et al. Conformance testing in systems with semicontrollable interfaces. Ann. Télécommun. 55, 70–83 (2000). https://doi.org/10.1007/BF02997773

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02997773

Key words

Mots clés

Search

Navigation