Formal Methods in System Design

, Volume 34, Issue 2, pp 183–213 | Cite as

Coverage-guided test generation for continuous and hybrid systems

  • Thao Dang
  • Tarik Nahhal


In this paper, we describe a formal framework for conformance testing of continuous and hybrid systems, using the international standard ‘Formal Methods in Conformance Testing’ FMCT. We propose a novel test coverage measure for these systems, which is defined using the star discrepancy notion. This coverage measure is used to quantify the validation ‘completeness’. It is also used to guide input stimulus generation by identifying the portions of the system behaviors that are not adequately examined. We then propose a test generation method, which is based on a robotic motion planning algorithm and is guided by the coverage measure. This method was implemented in a prototype tool that can handle high dimensional systems (up to 100 dimensions).


Hybrid systems Model-based testing Conformance testing Test coverage Test generation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alur R, Courcoubetis C, Halbwachs N, Henzinger TA, Ho P-H, Nicollin X, Olivero A, Sifakis J, Yovine S (1995) The algorithmic analysis of hybrid systems. Theor Comput Sci 138:3–34 zbMATHCrossRefGoogle Scholar
  2. 2.
    Alur R, Dang T, Esposito J, Hur Y, Ivancic F, Kumar V, Lee I, Mishra P, Pappas G, Sokolsky O (2002) Hierarchical modeling and analysis of embedded systems. In: Proceedings of the IEEE, October 2002 Google Scholar
  3. 3.
    Beck J, Chen WWL (1988) Irregularities of distribution. Cambridge Univ Press, Cambridge Google Scholar
  4. 4.
    Branicky MS, Curtiss MM, Levine J Morgan S (2005) Sampling-based reachability algorithms for control and verification of complex systems. In: Proc thirteenth Yale workshop on Adaptive and Learning Systems, New Haven, CT Google Scholar
  5. 5.
    Cheng P, LaValle S (2002) Resolution complete rapidly-exploring random trees. In: Proc IEEE International Conference on Robotics and Automation, pp 267–272 Google Scholar
  6. 6.
    Dang T, Nahhal T (2006) Randomized simulation of hybrid systems. Technical report, Verimag, May 2006 Google Scholar
  7. 7.
    Dobkin D, Eppstein D (1993) Computing the discrepancy. In: Proceedings of the Ninth Annual Symposium on Computational Geometry, pp 47–52 Google Scholar
  8. 8.
    Esposito JM, Kim J, Kumar V (2004) Adaptive RRTs for validating hybrid robotic control systems. In: Int workshop on the Algorithmic Foundations of Robotics Google Scholar
  9. 9.
    Bhatia A, Frazzoli E (2004) Incremental search methods for reachability analysis of continuous and hybrid systems. In: Hybrid Systems: Computation and Control HSCC. LNCS, vol 2993. Springer, Berlin, pp 142–156 Google Scholar
  10. 10.
    Julius AA, Fainekos GE, Anand M, Lee I, Pappas GJ (2007) Robust test generation and coverage for hybrid systems. In: Hybrid Systems: Computation and Control HSCC. LNCS. Springer, Berlin, pp 329–342 CrossRefGoogle Scholar
  11. 11.
    Kim J, Esposito JM, Kumar V (2006) Sampling-based algorithm for testing and validating robot controllers. Int J Rob Res 25(12):1257–1272 CrossRefGoogle Scholar
  12. 12.
    LaValle SM, Branicky MS, Lindemann SR (2004) On the relationship between classical grid search and probabilistic roadmaps. Int J Rob Res 23(7–8):673–692 CrossRefGoogle Scholar
  13. 13.
    Faure H (1978) Discrépance de suites associees à un système de numération. General theory of distribution modulo 1. In: Irregularities of distribution Google Scholar
  14. 14.
    Kuffner J, LaValle S (2000) RRT-connect: An efficient approach to single-query path planning. In Proc IEEE Int’l Conf on Robotics and Automation (ICRA’2000), San Francisco, CA, April 2000 Google Scholar
  15. 15.
    LaValle SM, Kuffner JJ (2001) Rapidly-exploring Random Trees: progress and prospects. In: Algorithmic and computational robotics: new directions. AK Peters, Wellesley, pp 293–308 Google Scholar
  16. 16.
    Lee D, Yannakakis M (1996) Principles and methods of testing finite state machines—A survey. In: Proceedings of the IEEE, vol 84, pp 1090–1123 Google Scholar
  17. 17.
    Lindemann SR, LaValle SM (2004) Incrementally reducing dispersion by increasing voronoi bias in RRTs. In: Proceedings IEEE International Conference on Robotics and Automation Google Scholar
  18. 18.
    Mitchell I, Tomlin C (2000) Level set methods for computation in hybrid Systems. In: Hybrid Systems: Control and Computation HSCC. LNCS, vol 1790. Springer, Berlin CrossRefGoogle Scholar
  19. 19.
    Moore A (1991) A tutorial on kd-trees. Computer laboratory technical report no 209, University of Cambridge,
  20. 20.
    Nahhal T, Dang T (2007) Guided randomized simulation. In: Hybrid Systems: Control and Computation. LNCS, vol 4416. Springer, Berlin, pp 731–735 CrossRefGoogle Scholar
  21. 21.
    Nahhal T (2007) Model-based testing of hybrid systems. PhD thesis, Joseph Fourier University, October 2007 Google Scholar
  22. 22.
    Plaku E, Kavraki LE, Vardi MY (2007) Hybrid systems: From verification to falsification. In: Damm W, Hermanns H (eds) International Conference on Computer Aided Verification (CAV). LNCS, vol 4590. Springer, Berlin, pp 468–481 CrossRefGoogle Scholar
  23. 23.
    Tan L, Kim J, Sokolsky O, Lee I (2004) Model-based testing and monitoring for hybrid embedded systems. In: IRI, pp 487–492 Google Scholar
  24. 24.
    Thiémard E (2001) An algorithm to compute bounds for the star discrepancy. J Complex 17(4):850 zbMATHCrossRefGoogle Scholar
  25. 25.
    Tretmans J (1999) Testing concurrent systems: a formal approach. In: Int Conference on Concurrency Theory CONCUR. LNCS, vol 1664. Springer, Berlin CrossRefGoogle Scholar
  26. 26.
    Tretmans J (1994) A formal approach to conformance testing. In: Proceedings of the IFIP TC6/WG6.1 sixth international workshop on Protocol Test Systems VI. North-Holland, Amsterdam, pp 257–276 Google Scholar
  27. 27.
    Yershova A, Jaillet L, Simeon T, LaValle SM (2005) Dynamic-domain RRTs: efficient exploration by controlling the sampling domain. In: Proc IEEE International Conference on Robotics and Automation Google Scholar
  28. 28.
    Wang X, Hickernell F (2000) Randomized Halton sequences. Math Comput Model 32:887–899 zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    Zhu H, Hall PAV, May JHR (1997) Software unit test coverage and adequacy. In: ACM Computing Surveys, 29, 4 Dec 1997 Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.VERIMAGGièresFrance

Personalised recommendations