Skip to main content
SpringerLink
Log in

Testing restorable systems: formal definition and heuristic solution based on river formation dynamics

  • Original Article
  • Published:
Formal Aspects of Computing

Abstract

Given a finite state machine denoting the specification of a system, finding some short interaction sequences capable of reaching some/all states or transitions of this machine is a typical goal in testing methods. If these sequences are applied to an implementation under test, then equivalent states or transitions would be reached and observed in the implementation—provided that the implementation were actually defined as the specification. We study the problem of finding such sequences in the case where configurations previously traversed can be saved and restored (at some cost). In general, this feature enables sequences to reach the required parts of the machine in less time, because some repetitions can be avoided. However, we show that finding optimal sequences in this case is an NP-hard problem. We propose an heuristic method to approximately solve this problem based on an evolutionary computation approach, in particular river formation dynamics (RFD). Given finite state machine specifications and sets of states/transitions to be reached, we apply RFD to construct testing plans reaching these configurations. Experimental results show that being able to load previously traversed states generally reduces the time needed to cover the target configurations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alba E, Chicano JF (2007) Ant colony optimization for model checking. In: EUROCAST’07. LNCS, vol 4739, pp 523–530

  2. Aho A, Dahbura A, Lee D, Uyar MÜ (1988) An optimization technique for protocol conformance test generation based on UIO sequences and Rural Chinese Postman tours. In: Protocol specification, testing and verification, PSTV’88. North-Holland, Amsterdam, pp 75–86

  3. Antoniol G, Di Penta M, Harman M (2004) A robust search-based approach to project management in the presence of abandonment, rework, error and uncertainty. In: IEEE software metrics symposium, METRICS’04. IEEE Computer Society, New York, pp 172–183

  4. Alba E, Troya JM (1996) Genetic algorithms for protocol validation. In: Parallel problem solving from nature, PPSN’96. LNCS, vol 1141. Springer, Berlin, pp 870–879

  5. Bernot G, Gaudel M-C, Marre B (1991) Software testing based on formal specification: a theory and a tool. Softw Eng J 6:387–405

    Google Scholar 

  6. Burgess CJ, Lefley M (2005) Can Genetic Programming improve Software Effort Estimation? A Comparative Evaluation. In: Machine learning applications in software engineering: series on software engineering and knowledge engineering, vol 16. World Scientific Publishing Co., Singapore, pp 95–105

  7. Bottaci L (2002) Instrumenting programs with flag variables for test data search by genetic algorithms. In: Conference on genetic and evolutionary computation, GECCO’02. Morgan Kaufmann Publishers Inc., Menlo Park, pp 1337–1342

  8. Brinksma E, Tretmans J (2001) Testing transition systems: an annotated bibliography. In: 4th summer school on modeling and verification of parallel processes, MOVEP’00. LNCS, vol 2067. Springer, Berlin, pp 187–195

  9. Chow TS (1978) Testing software design modeled by finite-state machines. IEEE Trans Softw Eng 4: 178–187

    Article  MATH  Google Scholar 

  10. Chen K, Jiang F, Huang C-D (2006) A new method of generating synchronizable test sequences that detect output-shifting faults based on multiple UIO sequences. In: Proceedings of the 2006 ACM symposium on applied computing, SAC’06. ACM, New York, pp 1791–1797

  11. Cohen M, Kooi SB, Srisa-an W (2006) Clustering the heap in multi-threaded applications for improved garbage collection. In: Conference on genetic and evolutionary computation, GECCO’06. ACM, New York, pp 1901–1908

  12. Dantzig GB (1963) Linear programming and extensions. Rand corporation research study. Princeton University Press, NJ

    Google Scholar 

  13. Doerner K, Gutjahr WJ (2003) Extracting test sequences from a Markov software usage model by ACO. In: Conference on genetic and evolutionary computation part II, GECCO’03. LNCS, vol 2724. Springer, Berlin, pp 2465–2476

  14. Derderian K, Hierons RM, Harman M, Guo Q (2006) Automated unique input output sequence generation for conformance testing of FSMs. Comput J 49(3): 331–344

    Article  Google Scholar 

  15. Dorigo M (2004) Ant colony optimization. MIT Press, Cambridge

    Book  MATH  Google Scholar 

  16. Dahbura A, Uyar MÜ (1995) Optimal test sequence generation for protocols: the Chinese Postman algorithm applied to Q.931. in: Conformance testing methodologies and architectures for OSI protocols. IEEE Computer Society Press, New York, pp 347–351

  17. Edelkamp S, Lluch-Lafuente A, Leue S (2001) Directed explicit model checking with HSF-SPIN. In: SPIN workshop on model checking of software, SPIN’01. LNCS, vol 2057. Springer, Berlin, pp 57–79

  18. En-Nouaary A, Dssouli R, Khendek F (2002) Timed wp-method: testing real-time systems. IEEE Trans Softw Eng 28: 1023–1038

    Article  Google Scholar 

  19. Gaudel M-C (1995) Testing can be formal, too. In: 6th CAAP/FASE, theory and practice of software development, TAPSOFT’95, LNCS, vol 915. Springer, Berlin, pp 82–96

  20. Godefroid P, Khurshid S (2002) Exploring very large state spaces using genetic algorithms. In: Conference on tools and algorithms for the construction and analysis of systems, TACAS’02. LNCS, vol 2280. Springer, Berlin, pp 266–280

  21. Gonenc G (1970) A method for the design of fault detection experiments. IEEE Trans Comput 19: 551–558

    Article  Google Scholar 

  22. Harman M (2007) The current state and future of search based software engineering. In: Workshop on the future of software engineering, FOSE’07, pp 342–357

  23. Hennie FC (1964) Fault detecting experiments for sequential circuits. In: Annual IEEE symposium on foundations of computer science, pp 95–110

  24. Harman M, Hierons RM, Proctor M (2002) A new representation and crossover operator for search-based optimization of software modularization. In: Conference on genetic and evolutionary computation, GECC0’02. Morgan Kaufmann Publishers, Menlo Park, pp 1351–1358 Inc

  25. Hierons RM (2002) Comparing test sets and criteria in the presence of test hypotheses and fault domains. ACM Trans Softw Eng Methodol 11(4): 427–448

    Article  Google Scholar 

  26. Hierons RM (2004) Testing from a nondeterministic finite state machine using adaptive state counting. IEEE Trans Comput 53: 1330–1342

    Article  Google Scholar 

  27. Hierons RM (2009) Verdict functions in testing with a fault domain or test hypotheses. ACM Trans Softw Eng Methodol 18(4): 1–19

    Article  Google Scholar 

  28. Harman M, McMinn P (2009) A theoretical and empirical study of search-based testing: local, global, and hybrid search. IEEE Trans Softw Eng 99(RapidPosts): 226–247

    Google Scholar 

  29. Holland JH (1975 )Adaptation in natural and artificial systems. MIT Press, Cambridge

    Google Scholar 

  30. Hierons RM, Ural H (2010) Generating a checking sequence with a minimum number of reset transitions. Autom Softw Eng 17: 217–250

    Article  Google Scholar 

  31. Huang G-D, Wang F (2005) Automatic test case generation with region-related coverage annotations for real-time systems. In: Symposium on automated technology for verification and analysis, ATVA’05. LNCS, vol 3707. Springer, Berlin, pp 144–158

  32. Ipate F, Banica L (2007) W-method for hierarchical and communicating finite state machines. In: IEEE international conference on industrial informatics, vol 2. IEEE Computer Society, New York, pp 891–896

  33. De Jong KA (2006) Evolutionary computation: a unified approach. MIT Press, Cambridge

    Google Scholar 

  34. Korel B (1992) Dynamic method for software test data generation. Softw Test Verif Reliab 2: 203–213

    Article  Google Scholar 

  35. Langdon WB, Harman M, Jia Y (2009) Multi-objective higher order mutation testing with genetic programming. In: Testing: academic and industrial conference—practice and research techniques, TAIC-PART’09. IEEE Computer Society, New York, pp 21–29

  36. Lam CP, Xiao J, Li H (2007) Ant colony optimisation for generation of conformance testing sequences using a characterising set. In: IASTED conference on advances in computer science and technology. ACTA Press, Mexico, pp 140–146

  37. Luo G, von Bochmann G, Petrenko A (1994) Test selection based on communicating nondeterministic finite-state machines using a generalized wp-method. IEEE Trans Softw Eng 20: 149–162

    Article  Google Scholar 

  38. Lawler EL, Wood DE (1966) Branch-and-bound methods: a survey. Oper Res 14(4): 699–719

    Article  MathSciNet  MATH  Google Scholar 

  39. Lee D, Yannakakis M (1994) Testing finite-state machines: state identification and verification. IEEE Trans Comput 43: 306–320

    Article  MathSciNet  Google Scholar 

  40. Lee D, Yannakakis M (1996) Principles and methods of testing finite state machines: a survey. Proc IEEE 84(8): 1090–1123

    Article  Google Scholar 

  41. Lehre PF, Yao X (2008) Crossover can be constructive when computing unique input output sequences. In: International conference on simulated evolution and learning, SEAL’08. LNCS, vol 5361. Springer, Berlin, pp 595–604

  42. Miller RE, Arisha KA (2001) Fault coverage in networks by passive testing. In: International conference on internet computing, IC’2001. CSREA Press, USA, pp 413–419

  43. McMinn P (2004) Search-based software test data generation: a survey. Softw Test Verif Reliab 14(2): 105–156

    Article  Google Scholar 

  44. Michael CC, Michael CC, Michael CC (2001) Generating software test data by evolution. IEEE Trans Softw Eng 27: 1085–1110

    Article  Google Scholar 

  45. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6): 1087–1092

    Article  Google Scholar 

  46. Petrenko A (2001) Fault model-driven test derivation from finite state models: Annotated bibliography. In: 4th summer school on modeling and verification of parallel processes, MOVEP’00. LNCS, vol 2067. Springer, Berlin, pp 196–205

  47. Pargas RP, Harrold MJ, Peck RR (1999) Test-data generation using genetic algorithms. Softw Test Verif Reliab 9: 263–282

    Article  Google Scholar 

  48. Petrenko A, Yevtushenko N, von Bochmann G (1996) Fault models for testing in context. In: Formal description techniques for distributed systems and communication protocols (IX), and protocol specification, testing, and verification (XVI). Chapman & Hall, London, pp 163–178

  49. Rodríguez I, Merayo MG, Núñez M (2008) \({\mathcal{HOTL}}\) : hypotheses and observations testing logic. J Log Algebr Program 74(2): 57–93

    Article  MATH  Google Scholar 

  50. Rodríguez I (2009) A general testability theory. In: International conference on concurrency theory, CONCUR’09. LNCS, vol 5710. Springer, Berlin, pp 572–586

  51. Rabanal P, Rodríguez I, Rubio F (2007) Using river formation dynamics to design heuristic algorithms. In: Unconventional computation, UC’07. LNCS, vol 4618. Springer, Berlin, pp 163–177

  52. Rabanal P, Rodríguez I, Rubio F (2008) Finding minimum spanning/distances trees by using river formation dynamics. In: Ant colony optimization and swarm intelligence, ANTS’08. LNCS, vol 5217. Springer, Berlin, pp 60–71

  53. Rabanal P, Rodríguez I, Rubio F (2009) Applying river formation dynamics to solve NP-complete problems. In: Chiong R (ed) Nature-inspired algorithms for optimisation. Studies in computational intelligence, vol 193. Springer, Berlin, pp 333–368

  54. Rabanal P, Rodríguez I, Rubio F (2009) A formal approach to heuristically test restorable systems. In: International colloquium on theoretical aspects of computing, ICTAC’09. LNCS, vol 5684. Springer, Berlin, pp 292–306

  55. Sabnani K, Dahbura A (1985) A new technique for generating protocol test. SIGCOMM Comput Commun Rev 15: 36–43

    Article  Google Scholar 

  56. Sabnani K, Dahbura A (1988) A protocol test generation procedure. Comput Netw ISDN Syst 15: 285–297

    Article  Google Scholar 

  57. Sidhu D, Leung T-K (1988) Fault coverage of protocol test methods. In: Networks: evolution or revolution. Seventh annual joint conference of the IEEE computer and communications societies, INFOCOM ’88. IEEE Computer Society, New York, pp 80–85

  58. Sidhu DP, Leung T (1989) Formal methods for protocol testing: a detailed study. IEEE Trans Softw Eng 15(4): 413–426

    Article  Google Scholar 

  59. Shen Y, Lombardi F (1996) Graph algorithms for conformance testing using the Rural Chinese Postman tour. SIAM J Discrete Math 9:511–528

    Google Scholar 

  60. Srivastava PR, Rai VK (2009) An ant colony optimization approach to test sequence generation for control flow based software testing. In: Information systems, technology and management. Communications in computer and information science, vol 31. Springer, Berlin, pp 345–346

  61. Tassey G (2002) The economic impacts of inadequate infrastructure for software testing. Technical report, National Institute of Standards and Technology

  62. Wang B, Hutchison D (1987) Protocol testing techniques. Comput Commun 10(2): 79–87

    Article  Google Scholar 

  63. Wegener J, Sthamer H, Jones BF, Eyres DE (1997) Testing real-time systems using genetic algorithms. Softw Qual Control 6: 127–135

    Article  Google Scholar 

  64. Zhu H, Hall PAV, May JHR (1997) Software unit test coverage and adequacy. ACM Comput Surv 29(4): 366–427

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. Sistemas Informáticos y Computación, Facultad de Informática, Universidad Complutense de Madrid, 28040, Madrid, Spain

    Pablo Rabanal, Ismael Rodríguez & Fernando Rubio

Authors
  1. Pablo Rabanal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ismael Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fernando Rubio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Rubio.

Additional information

Jim Woodcock

This paper constitutes and extended and revised version of [RRR09b]. Research partially supported by projects TIN2009-14312-C02-01 and Santander GR35/10-A—group number 910606.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rabanal, P., Rodríguez, I. & Rubio, F. Testing restorable systems: formal definition and heuristic solution based on river formation dynamics. Form Asp Comp 25, 743–768 (2013). https://doi.org/10.1007/s00165-011-0206-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00165-011-0206-3

Keywords

Search

Navigation