Runtime Verification with State Estimation

  • Scott D. Stoller
  • Ezio Bartocci
  • Justin Seyster
  • Radu Grosu
  • Klaus Havelund
  • Scott A. Smolka
  • Erez Zadok
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7186)


We introduce the concept of Runtime Verification with State Estimation and show how this concept can be applied to estimate the probability that a temporal property is satisfied by a run of a program when monitoring overhead is reduced by sampling. In such situations, there may be gaps in the observed program executions, thus making accurate estimation challenging. To deal with the effects of sampling on runtime verification, we view event sequences as observation sequences of a Hidden Markov Model (HMM), use an HMM model of the monitored program to “fill in” sampling-induced gaps in observation sequences, and extend the classic forward algorithm for HMM state estimation (which determines the probability of a state sequence, given an observation sequence) to compute the probability that the property is satisfied by an execution of the program. To validate our approach, we present a case study based on the mission software for a Mars rover. The results of our case study demonstrate high prediction accuracy for the probabilities computed by our algorithm. They also show that our technique is much more accurate than simply evaluating the temporal property on the given observation sequences, ignoring the gaps.


Hide Markov Model State Estimation Temporal Property Observation Sequence Property Instance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arnold, M., Vechev, M., Yahav, E.: QVM: An efficient runtime for detecting defects in deployed systems. In: Proc. 23rd ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2008), pp. 143–162. ACM (October 2008)Google Scholar
  2. 2.
    Baier, C., Katoen, J.P.: Principles of Model Checking. MIT Press (2008)Google Scholar
  3. 3.
    Barringer, H., Groce, A., Havelund, K., Smith, M.: Formal analysis of log files. Journal of Aerospace Computing, Information, and Communication 7(11), 365–390 (2010)Google Scholar
  4. 4.
    Barringer, H., Havelund, K.: TraceContract: A Scala DSL for Trace Analysis. In: Butler, M., Schulte, W. (eds.) FM 2011. LNCS, vol. 6664, pp. 57–72. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Bonakdarpour, B., Navabpour, S., Fischmeister, S.: Sampling-Based Runtime Verification. In: Butler, M., Schulte, W. (eds.) FM 2011. LNCS, vol. 6664, pp. 88–102. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  6. 6.
    Buss, E., Henshaw, J.: Experiences in program understanding. In: Proc. Second Conference of the Centre for Advanced Studies on Collaborative Research (CASCON 1992), pp. 157–189. IBM Press (1992)Google Scholar
  7. 7.
    Colombo, C., Gauci, A., Pace, G.J.: LarvaStat: Monitoring of Statistical Properties. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace, G., Roşu, G., Sokolsky, O., Tillmann, N. (eds.) RV 2010. LNCS, vol. 6418, pp. 480–484. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Diehl, S.: Software Visualization: Visualizing the Structure, Behavior, and Evolution of Software. Springer, Heidelberg (2007)Google Scholar
  9. 9.
    Fei, L., Midkiff, S.P.: Artemis: Practical runtime monitoring of applications for execution anomalies. In: Proc. 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2006), pp. 84–95. ACM, Ottawa (2006)CrossRefGoogle Scholar
  10. 10.
    Finkbeiner, B., Sankaranarayanan, S., Sipma, H.B.: Collecting statistics over runtime executions. Form. Methods Syst. Des. 27, 253–274 (2005)zbMATHCrossRefGoogle Scholar
  11. 11.
    Grunske, L.: An effective sequential statistical test for probabilistic monitoring. Information and Software Technology 53, 190–199 (2011)CrossRefGoogle Scholar
  12. 12.
    Hangal, S., Lam, M.S.: Tracking down software bugs using automatic anomaly detection. In: Proc. 24th International Conference on Software Engineering (ICSE 2002), pp. 291–301. ACM (2002)Google Scholar
  13. 13.
    Hauswirth, M., Chilimbi, T.M.: Low-overhead memory leak detection using adaptive statistical profiling. In: Proc. 11th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2004), pp. 156–164 (October 2004)Google Scholar
  14. 14.
    Huang, X., Seyster, J., Callanan, S., Dixit, K., Grosu, R., Smolka, S.A., Stoller, S.D., Zadok, E.: Software monitoring with controllable overhead. International Journal on Software Tools for Technology Transfer (2011)Google Scholar
  15. 15.
    Kwiatkowska, M., Norman, G., Parker, D.: Stochastic Model Checking. In: Bernardo, M., Hillston, J. (eds.) SFM 2007. LNCS, vol. 4486, pp. 220–270. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  16. 16.
    Lee, C., Chen, F., Roşu, G.: Mining parametric specifications. In: Proc. 33rd International Conference on Software Engineering (ICSE 2011), pp. 591–600. ACM (2011)Google Scholar
  17. 17.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  18. 18.
    Sammapun, U., Lee, I., Sokolsky, O., Regehr, J.: Statistical Runtime Checking of Probabilistic Properties. In: Sokolsky, O., Taşıran, S. (eds.) RV 2007. LNCS, vol. 4839, pp. 164–175. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  19. 19.
    Wang, Z., Zaki, M., Tahar, S.: Statistical runtime verification of analog and mixed signal designs. In: Proc. Third International Conference on Signals, Circuits and Systems (SCS 2009), pp. 1–6. IEEE (November 2009)Google Scholar
  20. 20.
    Zhang, L., Hermanns, H., Jansen, D.N.: Logic and Model Checking for Hidden Markov Models. In: Wang, F. (ed.) FORTE 2005. LNCS, vol. 3731, pp. 98–112. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  21. 21.
    Zhang, P., Ki, W., Wan, D., Grunske, L.: Monitoring of probabilistic timed property sequence charts. Software: Practice and Experience 41, 841–866 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Scott D. Stoller
    • 1
  • Ezio Bartocci
    • 2
  • Justin Seyster
    • 1
  • Radu Grosu
    • 1
  • Klaus Havelund
    • 3
  • Scott A. Smolka
    • 1
  • Erez Zadok
    • 1
  1. 1.Department of Computer ScienceStony Brook UniversityUSA
  2. 2.Department of Applied Math and StatisticsStony Brook UniversityUSA
  3. 3.Jet Propulsion LaboratoryCalifornia Institute of TechnologyUSA

Personalised recommendations