Advertisement

Formal Aspects of Computing

, Volume 25, Issue 5, pp 743–768 | Cite as

Testing restorable systems: formal definition and heuristic solution based on river formation dynamics

  • Pablo Rabanal
  • Ismael Rodríguez
  • Fernando RubioEmail author
Original Article

Abstract

Given a finite state machine denoting the specification of a system, finding some short interaction sequences capable of reaching some/all states or transitions of this machine is a typical goal in testing methods. If these sequences are applied to an implementation under test, then equivalent states or transitions would be reached and observed in the implementation—provided that the implementation were actually defined as the specification. We study the problem of finding such sequences in the case where configurations previously traversed can be saved and restored (at some cost). In general, this feature enables sequences to reach the required parts of the machine in less time, because some repetitions can be avoided. However, we show that finding optimal sequences in this case is an NP-hard problem. We propose an heuristic method to approximately solve this problem based on an evolutionary computation approach, in particular river formation dynamics (RFD). Given finite state machine specifications and sets of states/transitions to be reached, we apply RFD to construct testing plans reaching these configurations. Experimental results show that being able to load previously traversed states generally reduces the time needed to cover the target configurations.

Keywords

Testing Weighted Finite State Machines Minimum Load Sequence Evolutionary Computation River Formation Dynamics 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. AC07.
    Alba E, Chicano JF (2007) Ant colony optimization for model checking. In: EUROCAST’07. LNCS, vol 4739, pp 523–530Google Scholar
  2. ADLU88.
    Aho A, Dahbura A, Lee D, Uyar MÜ (1988) An optimization technique for protocol conformance test generation based on UIO sequences and Rural Chinese Postman tours. In: Protocol specification, testing and verification, PSTV’88. North-Holland, Amsterdam, pp 75–86Google Scholar
  3. APH04.
    Antoniol G, Di Penta M, Harman M (2004) A robust search-based approach to project management in the presence of abandonment, rework, error and uncertainty. In: IEEE software metrics symposium, METRICS’04. IEEE Computer Society, New York, pp 172–183Google Scholar
  4. AT96.
    Alba E, Troya JM (1996) Genetic algorithms for protocol validation. In: Parallel problem solving from nature, PPSN’96. LNCS, vol 1141. Springer, Berlin, pp 870–879Google Scholar
  5. BGM91.
    Bernot G, Gaudel M-C, Marre B (1991) Software testing based on formal specification: a theory and a tool. Softw Eng J 6:387–405Google Scholar
  6. BL05.
    Burgess CJ, Lefley M (2005) Can Genetic Programming improve Software Effort Estimation? A Comparative Evaluation. In: Machine learning applications in software engineering: series on software engineering and knowledge engineering, vol 16. World Scientific Publishing Co., Singapore, pp 95–105Google Scholar
  7. Bot02.
    Bottaci L (2002) Instrumenting programs with flag variables for test data search by genetic algorithms. In: Conference on genetic and evolutionary computation, GECCO’02. Morgan Kaufmann Publishers Inc., Menlo Park, pp 1337–1342Google Scholar
  8. BT01.
    Brinksma E, Tretmans J (2001) Testing transition systems: an annotated bibliography. In: 4th summer school on modeling and verification of parallel processes, MOVEP’00. LNCS, vol 2067. Springer, Berlin, pp 187–195Google Scholar
  9. [Cho78.
    Chow TS (1978) Testing software design modeled by finite-state machines. IEEE Trans Softw Eng 4: 178–187zbMATHCrossRefGoogle Scholar
  10. CJH06.
    Chen K, Jiang F, Huang C-D (2006) A new method of generating synchronizable test sequences that detect output-shifting faults based on multiple UIO sequences. In: Proceedings of the 2006 ACM symposium on applied computing, SAC’06. ACM, New York, pp 1791–1797Google Scholar
  11. CKSa06.
    Cohen M, Kooi SB, Srisa-an W (2006) Clustering the heap in multi-threaded applications for improved garbage collection. In: Conference on genetic and evolutionary computation, GECCO’06. ACM, New York, pp 1901–1908Google Scholar
  12. Dan63.
    Dantzig GB (1963) Linear programming and extensions. Rand corporation research study. Princeton University Press, NJGoogle Scholar
  13. DG03.
    Doerner K, Gutjahr WJ (2003) Extracting test sequences from a Markov software usage model by ACO. In: Conference on genetic and evolutionary computation part II, GECCO’03. LNCS, vol 2724. Springer, Berlin, pp 2465–2476Google Scholar
  14. DHHG06.
    Derderian K, Hierons RM, Harman M, Guo Q (2006) Automated unique input output sequence generation for conformance testing of FSMs. Comput J 49(3): 331–344CrossRefGoogle Scholar
  15. Dor04.
    Dorigo M (2004) Ant colony optimization. MIT Press, CambridgezbMATHCrossRefGoogle Scholar
  16. DU95.
    Dahbura A, Uyar MÜ (1995) Optimal test sequence generation for protocols: the Chinese Postman algorithm applied to Q.931. in: Conformance testing methodologies and architectures for OSI protocols. IEEE Computer Society Press, New York, pp 347–351Google Scholar
  17. ELLL01.
    Edelkamp S, Lluch-Lafuente A, Leue S (2001) Directed explicit model checking with HSF-SPIN. In: SPIN workshop on model checking of software, SPIN’01. LNCS, vol 2057. Springer, Berlin, pp 57–79Google Scholar
  18. ENDK02.
    En-Nouaary A, Dssouli R, Khendek F (2002) Timed wp-method: testing real-time systems. IEEE Trans Softw Eng 28: 1023–1038CrossRefGoogle Scholar
  19. Gau95.
    Gaudel M-C (1995) Testing can be formal, too. In: 6th CAAP/FASE, theory and practice of software development, TAPSOFT’95, LNCS, vol 915. Springer, Berlin, pp 82–96Google Scholar
  20. GK02.
    Godefroid P, Khurshid S (2002) Exploring very large state spaces using genetic algorithms. In: Conference on tools and algorithms for the construction and analysis of systems, TACAS’02. LNCS, vol 2280. Springer, Berlin, pp 266–280Google Scholar
  21. Gon70.
    Gonenc G (1970) A method for the design of fault detection experiments. IEEE Trans Comput 19: 551–558CrossRefGoogle Scholar
  22. Har07.
    Harman M (2007) The current state and future of search based software engineering. In: Workshop on the future of software engineering, FOSE’07, pp 342–357Google Scholar
  23. Hen64.
    Hennie FC (1964) Fault detecting experiments for sequential circuits. In: Annual IEEE symposium on foundations of computer science, pp 95–110Google Scholar
  24. HHP02.
    Harman M, Hierons RM, Proctor M (2002) A new representation and crossover operator for search-based optimization of software modularization. In: Conference on genetic and evolutionary computation, GECC0’02. Morgan Kaufmann Publishers, Menlo Park, pp 1351–1358 IncGoogle Scholar
  25. Hie02.
    Hierons RM (2002) Comparing test sets and criteria in the presence of test hypotheses and fault domains. ACM Trans Softw Eng Methodol 11(4): 427–448CrossRefGoogle Scholar
  26. Hie04.
    Hierons RM (2004) Testing from a nondeterministic finite state machine using adaptive state counting. IEEE Trans Comput 53: 1330–1342CrossRefGoogle Scholar
  27. Hie09.
    Hierons RM (2009) Verdict functions in testing with a fault domain or test hypotheses. ACM Trans Softw Eng Methodol 18(4): 1–19CrossRefGoogle Scholar
  28. HM09.
    Harman M, McMinn P (2009) A theoretical and empirical study of search-based testing: local, global, and hybrid search. IEEE Trans Softw Eng 99(RapidPosts): 226–247Google Scholar
  29. Hol75.
    Holland JH (1975 )Adaptation in natural and artificial systems. MIT Press, CambridgeGoogle Scholar
  30. HU10.
    Hierons RM, Ural H (2010) Generating a checking sequence with a minimum number of reset transitions. Autom Softw Eng 17: 217–250CrossRefGoogle Scholar
  31. HW05.
    Huang G-D, Wang F (2005) Automatic test case generation with region-related coverage annotations for real-time systems. In: Symposium on automated technology for verification and analysis, ATVA’05. LNCS, vol 3707. Springer, Berlin, pp 144–158Google Scholar
  32. IB07.
    Ipate F, Banica L (2007) W-method for hierarchical and communicating finite state machines. In: IEEE international conference on industrial informatics, vol 2. IEEE Computer Society, New York, pp 891–896Google Scholar
  33. Jon06.
    De Jong KA (2006) Evolutionary computation: a unified approach. MIT Press, CambridgeGoogle Scholar
  34. Kor92.
    Korel B (1992) Dynamic method for software test data generation. Softw Test Verif Reliab 2: 203–213CrossRefGoogle Scholar
  35. LHJ09.
    Langdon WB, Harman M, Jia Y (2009) Multi-objective higher order mutation testing with genetic programming. In: Testing: academic and industrial conference—practice and research techniques, TAIC-PART’09. IEEE Computer Society, New York, pp 21–29Google Scholar
  36. LJL07.
    Lam CP, Xiao J, Li H (2007) Ant colony optimisation for generation of conformance testing sequences using a characterising set. In: IASTED conference on advances in computer science and technology. ACTA Press, Mexico, pp 140–146Google Scholar
  37. LvBP94.
    Luo G, von Bochmann G, Petrenko A (1994) Test selection based on communicating nondeterministic finite-state machines using a generalized wp-method. IEEE Trans Softw Eng 20: 149–162CrossRefGoogle Scholar
  38. LW66.
    Lawler EL, Wood DE (1966) Branch-and-bound methods: a survey. Oper Res 14(4): 699–719MathSciNetzbMATHCrossRefGoogle Scholar
  39. LY94.
    Lee D, Yannakakis M (1994) Testing finite-state machines: state identification and verification. IEEE Trans Comput 43: 306–320MathSciNetCrossRefGoogle Scholar
  40. LY96.
    Lee D, Yannakakis M (1996) Principles and methods of testing finite state machines: a survey. Proc IEEE 84(8): 1090–1123CrossRefGoogle Scholar
  41. LY08.
    Lehre PF, Yao X (2008) Crossover can be constructive when computing unique input output sequences. In: International conference on simulated evolution and learning, SEAL’08. LNCS, vol 5361. Springer, Berlin, pp 595–604Google Scholar
  42. MA01.
    Miller RE, Arisha KA (2001) Fault coverage in networks by passive testing. In: International conference on internet computing, IC’2001. CSREA Press, USA, pp 413–419Google Scholar
  43. McM04.
    McMinn P (2004) Search-based software test data generation: a survey. Softw Test Verif Reliab 14(2): 105–156CrossRefGoogle Scholar
  44. MMS01.
    Michael CC, Michael CC, Michael CC (2001) Generating software test data by evolution. IEEE Trans Softw Eng 27: 1085–1110CrossRefGoogle Scholar
  45. MRR+53.
    Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6): 1087–1092CrossRefGoogle Scholar
  46. Pet01.
    Petrenko A (2001) Fault model-driven test derivation from finite state models: Annotated bibliography. In: 4th summer school on modeling and verification of parallel processes, MOVEP’00. LNCS, vol 2067. Springer, Berlin, pp 196–205Google Scholar
  47. PHP99.
    Pargas RP, Harrold MJ, Peck RR (1999) Test-data generation using genetic algorithms. Softw Test Verif Reliab 9: 263–282CrossRefGoogle Scholar
  48. PYB96.
    Petrenko A, Yevtushenko N, von Bochmann G (1996) Fault models for testing in context. In: Formal description techniques for distributed systems and communication protocols (IX), and protocol specification, testing, and verification (XVI). Chapman & Hall, London, pp 163–178Google Scholar
  49. RMN08.
    Rodríguez I, Merayo MG, Núñez M (2008) \({\mathcal{HOTL}}\) : hypotheses and observations testing logic. J Log Algebr Program 74(2): 57–93zbMATHCrossRefGoogle Scholar
  50. Rod09.
    Rodríguez I (2009) A general testability theory. In: International conference on concurrency theory, CONCUR’09. LNCS, vol 5710. Springer, Berlin, pp 572–586Google Scholar
  51. RRR07.
    Rabanal P, Rodríguez I, Rubio F (2007) Using river formation dynamics to design heuristic algorithms. In: Unconventional computation, UC’07. LNCS, vol 4618. Springer, Berlin, pp 163–177Google Scholar
  52. RRR08.
    Rabanal P, Rodríguez I, Rubio F (2008) Finding minimum spanning/distances trees by using river formation dynamics. In: Ant colony optimization and swarm intelligence, ANTS’08. LNCS, vol 5217. Springer, Berlin, pp 60–71Google Scholar
  53. RRR09a.
    Rabanal P, Rodríguez I, Rubio F (2009) Applying river formation dynamics to solve NP-complete problems. In: Chiong R (ed) Nature-inspired algorithms for optimisation. Studies in computational intelligence, vol 193. Springer, Berlin, pp 333–368Google Scholar
  54. RRR09b.
    Rabanal P, Rodríguez I, Rubio F (2009) A formal approach to heuristically test restorable systems. In: International colloquium on theoretical aspects of computing, ICTAC’09. LNCS, vol 5684. Springer, Berlin, pp 292–306Google Scholar
  55. SD85.
    Sabnani K, Dahbura A (1985) A new technique for generating protocol test. SIGCOMM Comput Commun Rev 15: 36–43CrossRefGoogle Scholar
  56. SD88.
    Sabnani K, Dahbura A (1988) A protocol test generation procedure. Comput Netw ISDN Syst 15: 285–297CrossRefGoogle Scholar
  57. SL88.
    Sidhu D, Leung T-K (1988) Fault coverage of protocol test methods. In: Networks: evolution or revolution. Seventh annual joint conference of the IEEE computer and communications societies, INFOCOM ’88. IEEE Computer Society, New York, pp 80–85Google Scholar
  58. SL89.
    Sidhu DP, Leung T (1989) Formal methods for protocol testing: a detailed study. IEEE Trans Softw Eng 15(4): 413–426CrossRefGoogle Scholar
  59. SL96.
    Shen Y, Lombardi F (1996) Graph algorithms for conformance testing using the Rural Chinese Postman tour. SIAM J Discrete Math 9:511–528Google Scholar
  60. SR09.
    Srivastava PR, Rai VK (2009) An ant colony optimization approach to test sequence generation for control flow based software testing. In: Information systems, technology and management. Communications in computer and information science, vol 31. Springer, Berlin, pp 345–346Google Scholar
  61. Tas02.
    Tassey G (2002) The economic impacts of inadequate infrastructure for software testing. Technical report, National Institute of Standards and TechnologyGoogle Scholar
  62. WH87.
    Wang B, Hutchison D (1987) Protocol testing techniques. Comput Commun 10(2): 79–87CrossRefGoogle Scholar
  63. WSJE97.
    Wegener J, Sthamer H, Jones BF, Eyres DE (1997) Testing real-time systems using genetic algorithms. Softw Qual Control 6: 127–135CrossRefGoogle Scholar
  64. ZHM97.
    Zhu H, Hall PAV, May JHR (1997) Software unit test coverage and adequacy. ACM Comput Surv 29(4): 366–427CrossRefGoogle Scholar

Copyright information

© BCS 2012

Authors and Affiliations

  • Pablo Rabanal
    • 1
  • Ismael Rodríguez
    • 1
  • Fernando Rubio
    • 1
    Email author
  1. 1.Dept. Sistemas Informáticos y Computación, Facultad de InformáticaUniversidad Complutense de MadridMadridSpain

Personalised recommendations