Model-Based Testing for General Stochastic Time

  • Marcus GerholdEmail author
  • Arnd Hartmanns
  • Mariëlle Stoelinga
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10811)


Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.


  1. 1.
    de Alfaro, L., Henzinger, T.A., Jhala, R.: Compositional methods for probabilistic systems. In: Larsen, K.G., Nielsen, M. (eds.) CONCUR 2001. LNCS, vol. 2154, pp. 351–365. Springer, Heidelberg (2001). CrossRefGoogle Scholar
  2. 2.
    Belinfante, A.: JTorX: a tool for on-line model-driven test derivation and execution. In: Esparza, J., Majumdar, R. (eds.) TACAS 2010. LNCS, vol. 6015, pp. 266–270. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  3. 3.
    Bohnenkamp, H., Belinfante, A.: Timed testing with TorX. In: Fitzgerald, J., Hayes, I.J., Tarlecki, A. (eds.) FM 2005. LNCS, vol. 3582, pp. 173–188. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  4. 4.
    Briones, L.B., Brinksma, E.: A test generation framework for quiescent real-time systems. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 64–78. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  5. 5.
    Cheung, L., Stoelinga, M., Vaandrager, F.: A testing scenario for probabilistic processes. J. ACM 54(6), 29 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Cheung, L., Lynch, N., Segala, R., Vaandrager, F.: Switched PIOA: parallel composition via distributed scheduling. Theor. Comput. Sci. 365(1), 83–108 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Cleaveland, R., Dayar, Z., Smolka, S.A., Yuen, S.: Testing preorders for probabilistic processes. Inf. Comput. 154(2), 93–148 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Conover, W.J.: A Kolmogorov goodness-of-fit test for discontinuous distributions. J. Am. Stat. Assoc. 67(339), 591–596 (1972)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    D’Argenio, P.R., Lee, M.D., Monti, R.E.: Input/output stochastic automata. In: Fränzle, M., Markey, N. (eds.) FORMATS 2016. LNCS, vol. 9884, pp. 53–68. Springer, Cham (2016). CrossRefGoogle Scholar
  10. 10.
    Deng, Y., Hennessy, M., van Glabbeek, R.J., Morgan, C.: Characterising testing preorders for finite probabilistic processes. CoRR (2008)Google Scholar
  11. 11.
    Duflot, M., Kwiatkowska, M., Norman, G., Parker, D.: A formal analysis of bluetooth device discovery. STTT 8(6), 621–632 (2006)CrossRefGoogle Scholar
  12. 12.
    Eisentraut, C., Hermanns, H., Zhang, L.: On probabilistic automata in continuous time. In: LICS, pp. 342–351. IEEE Computer Society (2010)Google Scholar
  13. 13.
    Gerhold, M., Stoelinga, M.: Model-based testing of probabilistic systems. In: Stevens, P., Wąsowski, A. (eds.) FASE 2016. LNCS, vol. 9633, pp. 251–268. Springer, Heidelberg (2016). CrossRefGoogle Scholar
  14. 14.
    Gerhold, M., Stoelinga, M.: Model-based testing of probabilistic systems with stochastic time. In: Gabmeyer, S., Johnsen, E.B. (eds.) TAP 2017. LNCS, vol. 10375, pp. 77–97. Springer, Cham (2017). CrossRefGoogle Scholar
  15. 15.
    Gordon, A.D., Henzinger, T.A., Nori, A.V., Rajamani, S.K.: Probabilistic programming. In: FOSE, pp. 167–181. ACM (2014)Google Scholar
  16. 16.
    Hierons, R.M., Merayo, M.G., Núñez, M.: Testing from a stochastic timed system with a fault model. J. Log. Algebr. Program. 78(2), 98–115 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Hollander, M., Wolfe, D.A., Chicken, E.: Nonparametric Statistical Methods. Wiley, Hoboken (2013)zbMATHGoogle Scholar
  18. 18.
    Katoen, J.P.: The probabilistic model checking landscape. In: LICS. ACM (2016)Google Scholar
  19. 19.
    Krichen, M., Tripakis, S.: Conformance testing for real-time systems. Form. Methods Syst. Des. 34(3), 238–304 (2009)CrossRefzbMATHGoogle Scholar
  20. 20.
    Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. ACM (1989)Google Scholar
  21. 21.
    Larsen, K.G., Mikucionis, M., Nielsen, B.: Online testing of real-time systems using Uppaal. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 79–94. Springer, Heidelberg (2005). CrossRefGoogle Scholar
  22. 22.
    Milner, R. (ed.): A Calculus of Communicating Systems. LNCS, vol. 92. Springer, Heidelberg (1980). zbMATHGoogle Scholar
  23. 23.
    Núñez, M., Rodríguez, I.: Towards testing stochastic timed systems. In: König, H., Heiner, M., Wolisz, A. (eds.) FORTE 2003. LNCS, vol. 2767, pp. 335–350. Springer, Heidelberg (2003). CrossRefGoogle Scholar
  24. 24.
    Segala, R.: Modeling and verification of randomized distributed real-time systems. Ph.D. thesis, Cambridge, MA, USA (1995)Google Scholar
  25. 25.
    Stoelinga, M.: Alea jacta est: verification of probabilistic, real-time and parametric systems. Ph.D. thesis, Radboud University of Nijmegen (2002)Google Scholar
  26. 26.
    Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT press, Cambridge (2005)zbMATHGoogle Scholar
  27. 27.
    Tretmans, J.: Conformance testing with labelled transition systems: implementation relations and test generation. Comput. Netw. ISDN Syst. 29(1), 49–79 (1996)CrossRefGoogle Scholar
  28. 28.
    Tretmans, J.: Model based testing with labelled transition systems. In: Hierons, R.M., Bowen, J.P., Harman, M. (eds.) Formal Methods and Testing. LNCS, vol. 4949, pp. 1–38. Springer, Heidelberg (2008). CrossRefGoogle Scholar
  29. 29.
    Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Test. Verif. Reliab. 22(5), 297–312 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.University of TwenteEnschedeThe Netherlands

Personalised recommendations