Test Coverage Estimation Using Threshold Accepting
This paper is concerned with model-based testing of hybrid systems. In our previous work , we proposed a test generation algorithm, called gRRT, guided by a coverage measure defined using the star discrepancy notion. An important ingredient in this algorithm is a procedure for dynamically estimating the coverage, which is done based on a box partition of the continuous state space. The goal of this estimation is to identify the areas in the state space which have not been sufficiently visited. A drawback of this guiding method is that its complexity depends on the number of the boxes in the partition, which needs to be fine enough to guarantee a good coverage estimate. Thus in high dimensions the method can become very costly. To enhance the scalability of the algorithm gRRT we propose in this paper a new guiding method, motivated by the observation that trying to optimize the coverage in each exploration step is, on one hand, computationally costly, and on the other hand, not always a good choice since this may make the system try to expand in the directions which are not reachable (due to the controllability of the system). Instead of considering all the boxes in the partition, we propose to use a randomized search to quickly find a region that yields a high local discrepancy value. This randomized search is based on threshold accepting, a well-known integer optimization heuristic. We also present some experimental results obtained on a challenging circuit benchmark and a number of randomly generated examples, which shows that the new guiding method allows achieving better time and coverage efficiency.
Unable to display preview. Download preview PDF.
- 1.Abbas, H., Fainekos, G.: Convergence Proofs for Simulated Annealing Falsification of Safety Properties. In: Proc. of 50th Annual Allerton Conference on Communication, Control, and Computing, Monticello, IL (October 2012)Google Scholar
- 3.Beck, J., Chen, W.W.L.: Irregularities of distribution. In: Acta Arithmetica, UK, Cambridge University Press (1997)Google Scholar
- 4.Branicky, M., Curtiss, M., Levine, J., Morgan, S.: Sampling-based reachability algorithms for control and verification of complex systems. In: Thirteenth Yale Workshop on Adaptive and Learning Systems (2005)Google Scholar
- 8.Dobkin, D., Eppstein, D.: Computing the discrepancy. In: SCG 1993: Proceedings of the Ninth Annual Symposium on Computational Geometry, pp. 47–52. ACM Press, New York (1993)Google Scholar
- 10.Esposito, J., Kim, J.W., Kumar, V.: Adaptive RRTs for validating hybrid robotic control systems. In: Proceedings Workshop on Algorithmic Foundations of Robotics, Zeist, The Netherlands (July 2004)Google Scholar
- 11.Bhatia, A., Frazzoli, E.: Incremental search methods for reachability analysis of continuous and hybrid systems. In: Alur, R., Pappas, G.J. (eds.) HSCC 2004. LNCS, vol. 2993, pp. 142–156. Springer, Heidelberg (2004)Google Scholar
- 13.Gnewuch, M., Wahlström, M., Winzen, C.: SIAM Journal of Numerical Analysis 50, 781–807 (2012)Google Scholar
- 14.Julius, A., Fainekos, G.E., Anand, M., Lee, I., Pappas, G.J.: Robust test generation and coverage for hybrid systems. In: Bemporad, A., Bicchi, A., Buttazzo, G. (eds.) HSCC 2007. LNCS, vol. 4416, pp. 329–342. Springer, Heidelberg (2007)Google Scholar
- 15.Kuffner, J., LaValle, S.: RRT-connect: An efficient approach to single-query path planning. In: Proc. IEEE Int’l Conf. on Robotics and Automation (ICRA 2000), San Francisco, CA (April 2000)Google Scholar
- 16.LaValle, S., Kuffner, J.: Rapidly-exploring random trees: Progress and prospects. In: Workshop on the Algorithmic Foundations of Robotics (2000)Google Scholar
- 17.Mcinn, P.: Search-based Software Test Data Generation: A Survey: Research Articles. Softw. Test. Verif. Reliab. 14(2), 105–156Google Scholar
- 20.Sankaranarayanan, S., Fainekos, G.: Falsification of Temporal Properties of Hybrid Systems Using the Cross-Entropy Method. In: Proceedings of the ACM International Conference on Hybrid Systems: Computation and Control, Beijing, China (April 2012)Google Scholar