Advertisement

Model-Based Testing for General Stochastic Time

  • Marcus Gerhold
  • Arnd Hartmanns
  • Mariëlle Stoelinga
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10811)

Abstract

Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.

References

  1. 1.
    de Alfaro, L., Henzinger, T.A., Jhala, R.: Compositional methods for probabilistic systems. In: Larsen, K.G., Nielsen, M. (eds.) CONCUR 2001. LNCS, vol. 2154, pp. 351–365. Springer, Heidelberg (2001).  https://doi.org/10.1007/3-540-44685-0_24 CrossRefGoogle Scholar
  2. 2.
    Belinfante, A.: JTorX: a tool for on-line model-driven test derivation and execution. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 266–270. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-12002-2_21 CrossRefGoogle Scholar
  3. 3.
    Bohnenkamp, H., Belinfante, A.: Timed testing with TorX. In: Fitzgerald, J., Hayes, I.J., Tarlecki, A. (eds.) FM 2005. LNCS, vol. 3582, pp. 173–188. Springer, Heidelberg (2005).  https://doi.org/10.1007/11526841_13 CrossRefGoogle Scholar
  4. 4.
    Briones, L.B., Brinksma, E.: A test generation framework for quiescent real-time systems. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 64–78. Springer, Heidelberg (2005).  https://doi.org/10.1007/978-3-540-31848-4_5 CrossRefGoogle Scholar
  5. 5.
    Cheung, L., Stoelinga, M., Vaandrager, F.: A testing scenario for probabilistic processes. J. ACM 54(6), 29 (2007)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Cheung, L., Lynch, N., Segala, R., Vaandrager, F.: Switched PIOA: parallel composition via distributed scheduling. Theor. Comput. Sci. 365(1), 83–108 (2006)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Cleaveland, R., Dayar, Z., Smolka, S.A., Yuen, S.: Testing preorders for probabilistic processes. Inf. Comput. 154(2), 93–148 (1999)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Conover, W.J.: A Kolmogorov goodness-of-fit test for discontinuous distributions. J. Am. Stat. Assoc. 67(339), 591–596 (1972)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    D’Argenio, P.R., Lee, M.D., Monti, R.E.: Input/output stochastic automata. In: Fränzle, M., Markey, N. (eds.) FORMATS 2016. LNCS, vol. 9884, pp. 53–68. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-44878-7_4 CrossRefGoogle Scholar
  10. 10.
    Deng, Y., Hennessy, M., van Glabbeek, R.J., Morgan, C.: Characterising testing preorders for finite probabilistic processes. CoRR (2008)Google Scholar
  11. 11.
    Duflot, M., Kwiatkowska, M., Norman, G., Parker, D.: A formal analysis of bluetooth device discovery. STTT 8(6), 621–632 (2006)CrossRefGoogle Scholar
  12. 12.
    Eisentraut, C., Hermanns, H., Zhang, L.: On probabilistic automata in continuous time. In: LICS, pp. 342–351. IEEE Computer Society (2010)Google Scholar
  13. 13.
    Gerhold, M., Stoelinga, M.: Model-based testing of probabilistic systems. In: Stevens, P., Wąsowski, A. (eds.) FASE 2016. LNCS, vol. 9633, pp. 251–268. Springer, Heidelberg (2016).  https://doi.org/10.1007/978-3-662-49665-7_15 CrossRefGoogle Scholar
  14. 14.
    Gerhold, M., Stoelinga, M.: Model-based testing of probabilistic systems with stochastic time. In: Gabmeyer, S., Johnsen, E.B. (eds.) TAP 2017. LNCS, vol. 10375, pp. 77–97. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-61467-0_5 CrossRefGoogle Scholar
  15. 15.
    Gordon, A.D., Henzinger, T.A., Nori, A.V., Rajamani, S.K.: Probabilistic programming. In: FOSE, pp. 167–181. ACM (2014)Google Scholar
  16. 16.
    Hierons, R.M., Merayo, M.G., Núñez, M.: Testing from a stochastic timed system with a fault model. J. Log. Algebr. Program. 78(2), 98–115 (2009)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Hollander, M., Wolfe, D.A., Chicken, E.: Nonparametric Statistical Methods. Wiley, Hoboken (2013)MATHGoogle Scholar
  18. 18.
    Katoen, J.P.: The probabilistic model checking landscape. In: LICS. ACM (2016)Google Scholar
  19. 19.
    Krichen, M., Tripakis, S.: Conformance testing for real-time systems. Form. Methods Syst. Des. 34(3), 238–304 (2009)CrossRefMATHGoogle Scholar
  20. 20.
    Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. ACM (1989)Google Scholar
  21. 21.
    Larsen, K.G., Mikucionis, M., Nielsen, B.: Online testing of real-time systems using Uppaal. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 79–94. Springer, Heidelberg (2005).  https://doi.org/10.1007/978-3-540-31848-4_6 CrossRefGoogle Scholar
  22. 22.
    Milner, R. (ed.): A Calculus of Communicating Systems. LNCS, vol. 92. Springer, Heidelberg (1980).  https://doi.org/10.1007/3-540-10235-3 MATHGoogle Scholar
  23. 23.
    Núñez, M., Rodríguez, I.: Towards testing stochastic timed systems. In: König, H., Heiner, M., Wolisz, A. (eds.) FORTE 2003. LNCS, vol. 2767, pp. 335–350. Springer, Heidelberg (2003).  https://doi.org/10.1007/978-3-540-39979-7_22 CrossRefGoogle Scholar
  24. 24.
    Segala, R.: Modeling and verification of randomized distributed real-time systems. Ph.D. thesis, Cambridge, MA, USA (1995)Google Scholar
  25. 25.
    Stoelinga, M.: Alea jacta est: verification of probabilistic, real-time and parametric systems. Ph.D. thesis, Radboud University of Nijmegen (2002)Google Scholar
  26. 26.
    Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT press, Cambridge (2005)MATHGoogle Scholar
  27. 27.
    Tretmans, J.: Conformance testing with labelled transition systems: implementation relations and test generation. Comput. Netw. ISDN Syst. 29(1), 49–79 (1996)CrossRefGoogle Scholar
  28. 28.
    Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-78917-8_1 CrossRefGoogle Scholar
  29. 29.
    Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22(5), 297–312 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of TwenteEnschedeThe Netherlands

Personalised recommendations