Applied Intelligence

, Volume 25, Issue 1, pp 107–126 | Cite as

Design of test inputs and their sequences in multi-function system testing

  • Mark Sh. LevinEmail author
  • Mark Last


This discussion paper addresses combinatorial models in system testing from the perspective of system usage (utilization) and corresponding examination of system functions and their groups. Thus the following aspects of multi-function system testing are under study: analysis of system requirements and revelation of atomic system functions and their relationships, analysis of system function groups (clusters), design of the most important test inputs and sequences of the test inputs. The basic combinatorial problem is: composition of the best (the most important) test input(s) for each group of atomic system functions. Additional combinatorial problems are the following: (a) design of test input sequence for a trail (chain) of function clusters, (b) design of collection of test input sequences as covering of function cluster digraph, (c) structural fusion of unit test results. Numerical and real world examples illustrate the proposed approach.


Test input Test inputs sequences Multi-function testing Black-box testing Design System approach Decision making Combinatorial optimization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Beizer B (1995) Black-Box testing: Techniques for Functional Testing of Software and Systems. Wiley, New YorkGoogle Scholar
  2. 2.
    Bogdanov K, Holombe M (2001) Statechart testing method for aircraft control systems. J Test Verifi Reliab 11(1):39–54CrossRefGoogle Scholar
  3. 3.
    Buede DM (1992) Software review. Overview of MCDA software market. J Multi-Crit Decision Anal 1(1):59–61Google Scholar
  4. 4.
    Burr K, Young W (1998) Combinatorial test techniques: Table-based automation, test generation, and test coverage. In: Proc Int Conf on Software Testing, Analysis, and Review (STAR). San Diego, CA, pp 26–28Google Scholar
  5. 5.
    Cameron K (1990) An algorithmic note on the Galai-Milgram theorem. Networks 20:43–48zbMATHMathSciNetGoogle Scholar
  6. 6.
    Chanrasekaran B, Josephson JR (2000) Function in dence representation. J Eng Compu 16(3/4):162–177CrossRefGoogle Scholar
  7. 7.
    Cheng KT, Jou JY (1990) Functional test generation for finite state machine. In: Proc IEEE Int Testing Conf, IEEE, New York, pp 162–168Google Scholar
  8. 8.
    Chittarq I, Rannon R (1999) Diagnosis of multiple faults with flow-based functional models: The functional diagnosis with efforts and flows approach. Reliab Eng Syst Saf 64(2):137–150CrossRefGoogle Scholar
  9. 9.
    Chow TS (1978) Testing design modeled by finite-state machines. IEEE Trans Softw Eng 4(3):178–186zbMATHGoogle Scholar
  10. 10.
    Clarke EM, Grumberg O, Peled DA (2000) Model Checking. MIT Press, Ca mbridge, MassGoogle Scholar
  11. 11.
    Cohen DM, Dalal SR, Parelius J, Patton GC (1996) The combinatorial design approach to automatic test generation. IEEE Softw pp 83–87Google Scholar
  12. 12.
    Diestel R (1997) Graph theory. Springer-Verlag, New YorkzbMATHGoogle Scholar
  13. 13.
    Edmonds J (1965) The Chinese postman’s problem, Bull the Oper Res Soc Amer 13(B-73):486–487 and 519Google Scholar
  14. 14.
    Garey MR, Johnson DS (1979) Computers and intractability. The Guide to the Theory of NP-completeness. Freeman, San FranciscoGoogle Scholar
  15. 15.
    Goldschmidt O, Hochbaum DS, Hurkens C, Yu G (1996) Approximation algorithms for (kmbox-)clique covering problems. SIAM J Discr Math 9(3):492–509MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Harary F (1969) Graph theory. Addison-Wesley, Reading, MassGoogle Scholar
  17. 17.
    IEEE Standard 610 (1990) IEEE Standard Collection: Software Engineering. IEEE, 1994Google Scholar
  18. 18.
    Jackson B (1981) tbl begintabulartoprule Long paths and cycles in oriented graphs. J Graph The 5:145–157zbMATHGoogle Scholar
  19. 19.
    Jorgensen PC (2002) Software Testing. A Craftman’s Approach, 2ed. CRC, Boca Raton, FLGoogle Scholar
  20. 20.
    Kaner C, Falk J, Nguyen HQ (1999) Testing Computer Software, 2nd ed. Wiley, New YorkGoogle Scholar
  21. 21.
    Kaner C (2003) What is a good test case? Software Testing Analysis & Review Conference (STAR) East, Orlando, FLGoogle Scholar
  22. 22.
    Karpovsky MG, Moskalev EA (1986) Covering of edges of graph by a minimal set of paths. Discr Math 58(2):214Google Scholar
  23. 23.
    Keeny RL, Raiffa H (1976) Decisions with multiple objectives: Preferences and value tradeoffs. Wiley, New YorkzbMATHGoogle Scholar
  24. 24.
    Korhonen P, Wallenius J, Zionts S (1984) Solving the discrete multiple criteria problems using convex cones. Manag Sci 30(11):1336–1345MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Lai K-W, Siewiorek DP (1983) Functional testing of digital systems. In: Proc 20th Design Automation Conf., IEEE, pp 207–213Google Scholar
  26. 26.
    Last M, Kandel A (2003) Automated test reduction using an Info-Fuzzy Network. In: Khoshgoftaar TM (ed) Software Engineering with Computational Intelligence, Kluwer, pp 235–258Google Scholar
  27. 27.
    Last M, Friedman M, Kandel A (2003) The data mining approach to automated software testing. In: Proc of the 9th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining (KDD-2003), Washington, DC, USA, pp 388–395Google Scholar
  28. 28.
    Levin MSh (1988) Typical Approach to Quality Evaluation in Machine-Building. VNIIKI, Moscow (in Russian)Google Scholar
  29. 29.
    Levin MSh (1998) Combinatorial Engineering of Decomposable Systems. Kluwer, DordrechtzbMATHGoogle Scholar
  30. 30.
    Levin MSh (2002) Towards combinatorial analysis, adaptation, and planning of human-computer systems. Appl Intell 16(3):235–247zbMATHCrossRefGoogle Scholar
  31. 31.
    Levin MSh, Last M (2004) Collection of test case sequences: Covering of function cluster digraph. In: Proc IASTED Conf on AI and Applications. Innsbruck, pp 806–810Google Scholar
  32. 32.
    Levin MSh, Last M (2004) Multi-function system testing: Composition of test sets, In: Proc. The 8th IEEE Int Symp on High Assurance Syst. Engineering “HASE’04”, IEEE Computer Society Press, pp 99–108Google Scholar
  33. 33.
    Levin MSh, Last M (2004) Test case sequences in system testing: Selection of test cases for a chain (sequence) of function clusters. In: Proc The 17th Int Conf IEA/AIE, Ottawa, LNCS 3029, Springer, pp 895–904Google Scholar
  34. 34.
    Leung KRPH, Wong W, Ng JK-Y (2003) Generating test cases from class vectors. J Syst Softw 66(1):35–46Google Scholar
  35. 35.
    Linial N (1978) Covering digraphs by paths. Discr Math 23(3):257–272zbMATHMathSciNetCrossRefGoogle Scholar
  36. 36.
    Lipaev VV (2003) A methodology of verification and testing of large software systems. Progr Comp Softw 29(6):298–309zbMATHCrossRefGoogle Scholar
  37. 37.
    Madrioli D, Morasca S, Morzenti A (1995) Generating test cases for real-time systems from logic specifications. ACM Trans. Comp Syst 13(4):365–398CrossRefGoogle Scholar
  38. 38.
    Martello S, Toth P (1990) Knapsack problem: Algorithms and Computer Implementation. Wiley, New YorkGoogle Scholar
  39. 39.
    Ostrand TJ, Balcer MJ (1988) The category-partition method for specifying and generating functional tests. Comm ACM 31(6):676–686CrossRefGoogle Scholar
  40. 40.
    Paton R (2001) Software Testing. Sams, Indianapolis, ILGoogle Scholar
  41. 41.
    Peled D, Vardi MY, Yannakakis M (2002) Black box checking. J Aut, Lang Comb 7(2):225–246MathSciNetzbMATHGoogle Scholar
  42. 42.
    Roy B (1990) The outranking Approach and Foundations of ELECTRE Methods. In: Bana e Costa CA (ed) Readings in Multi-Criteria Decision Aid. Springer-Verlag, Berlin. pp 155–183Google Scholar
  43. 43.
    Saaty TL (1988) The Analytic Hierarchy Process. MacGraw-Hill, New YorkGoogle Scholar
  44. 44.
    Schroeder PJ, Korel B (2000) Black-box test reduction using Input-output analysis. ACM SIGSOFT Softw Eng Notes 25(5):173–177CrossRefGoogle Scholar
  45. 45.
    Shenhar AJ (1998) From theory to practice: Toward a typology of project-management styles. IEEE Trans Eng Manag 45(1):33–48CrossRefGoogle Scholar
  46. 46.
    Steuer RL (1986) Multiple criteria optimization: Theory, computation, and Application. Wiley, New YorkzbMATHGoogle Scholar
  47. 47.
    Stoica S (1999) Generating functional design verification tests. IEEE Design & Test 16(3):53-63CrossRefGoogle Scholar
  48. 48.
    Tamres V (2002) Introducing testing software. Addison-Wesley, MassGoogle Scholar
  49. 49.
    Turing A (1936) On computable numbers with an application to the Enscheidungsproblem. In: Proc of Lond Math Soc, vol XLII, pp 239–265 (with correction in ibid, vol XLIII, 1937, pp. 544–546)Google Scholar
  50. 50.
    Turner CD, Robson DJ (1993) State-based testing and inheritance. Technical Report TR1/93, Univ of Durham, EnglandGoogle Scholar
  51. 51.
    Volkmann L (1999) Longest path in semicomplete multipartite digraphs. Discr Math 199(1–3):279–284zbMATHMathSciNetGoogle Scholar
  52. 52.
    Wilson RJ (1972) Introduction to graph theory. Oliver and Boyd, EdinburghzbMATHGoogle Scholar
  53. 53.
    Ural H, Zhu K (1993) Optimal length test sequence generation using distinguishing sequences. IEEE Trans Netw 1(3):358–371CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Department of Information System EngineeringBen-Gurion UniversityBeer ShevaIsrael
  2. 2.Institute for Information Transmission ProblemsRussian Academy of SciencesMoscowRussia

Personalised recommendations