Synthesizing Probabilistic Invariants via Doob’s Decomposition

  • Gilles BartheEmail author
  • Thomas Espitau
  • Luis María Ferrer Fioriti
  • Justin Hsu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9779)


When analyzing probabilistic computations, a powerful approach is to first find a martingale—an expression on the program variables whose expectation remains invariant—and then apply the optional stopping theorem in order to infer properties at termination time. One of the main challenges, then, is to systematically find martingales.

We propose a novel procedure to synthesize martingale expressions from an arbitrary initial expression. Contrary to state-of-the-art approaches, we do not rely on constraint solving. Instead, we use a symbolic construction based on Doob’s decomposition. This procedure can produce very complex martingales, expressed in terms of conditional expectations.

We show how to automatically generate and simplify these martingales, as well as how to apply the optional stopping theorem to infer properties at termination time. This last step typically involves some simplification steps, and is usually done manually in current approaches. We implement our techniques in a prototype tool and demonstrate our process on several classical examples. Some of them go beyond the capability of current semi-automatic approaches.


Conditional Expectation Program Variable Probabilistic Program Proof Rule Symbolic Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We thank the anonymous reviewers for their helpful comments. This work was partially supported by NSF grants TWC-1513694 and CNS-1065060, and by a grant from the Simons Foundation (\(\#360368\) to Justin Hsu).


  1. 1.
    Allen, J.R., Kennedy, K., Porterfield, C., Warren, J.: Conversion of control dependence to data dependence. In: ACM Symposium on Principles of Programming Languages (POPL), Austin, Texas, pp. 177–189. ACM, New York (1983). ISBN 0-89791-090-7Google Scholar
  2. 2.
    Ammarguellat, Z., Harrison III, W.L.: Automatic recognition of induction variables and recurrence relations by abstract interpretation. In: ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), White Plains, New York, pp. 283–295 (1990)Google Scholar
  3. 3.
    Baier, C., Katoen, J.: Principles of Model Checking. MIT Press, Cambridge (2008). ISBN 978-0-262-02649-9Google Scholar
  4. 4.
    Barthe, G., Grégoire, B., Heraud, S., Béguelin, S.Z.: Computer-aided security proofs for the working cryptographer. In: Rogaway, P. (ed.) CRYPTO 2011. LNCS, vol. 6841, pp. 71–90. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Bournez, O., Garnier, F.: Proving positive almost-sure termination. In: Giesl, J. (ed.) RTA 2005. LNCS, vol. 3467, pp. 323–337. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Chakarov, A., Sankaranarayanan, S.: Probabilistic program analysis with martingales. In: Sharygina, N., Veith, H. (eds.) CAV 2013. LNCS, vol. 8044, pp. 511–526. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  7. 7.
    Chakarov, A., Sankaranarayanan, S.: Expectation invariants for probabilistic program loops as fixed points. In: Müller-Olm, M., Seidl, H. (eds.) Static Analysis. LNCS, vol. 8723, pp. 85–100. Springer, Heidelberg (2014)Google Scholar
  8. 8.
    Chakarov, A., Voronin, Y.-L., Sankaranarayanan, S.: Deductive proofs of almost sure persistence and recurrence properties. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 260–279. Springer, Heidelberg (2016). doi: 10.1007/978-3-662-49674-9_15 CrossRefGoogle Scholar
  9. 9.
    Chatterjee, K., Fu, H., Novotný, P., Hasheminezhad, R.: Algorithmic analysis of qualitative and quantitative termination problems for affine probabilistic programs. In: ACM SIGPLAN–SIGACT Symposium on Principles of Programming Languages (POPL), St. Petersburg, Florida, pp. 327–342 (2016)Google Scholar
  10. 10.
    Chen, Y.-F., Hong, C.-D., Wang, B.-Y., Zhang, L.: Counterexample-guided polynomial loop invariant generation by lagrange interpolation. In: Kroening, D., Păsăreanu, C.S. (eds.) CAV 2015. LNCS, vol. 9206, pp. 658–674. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  11. 11.
    Ciesinski, F., Baier, C.: LiQuor: a tool for qualitative and quantitative linear time analysis of reactive systems. In: Third International Conference on the Quantitative Evaluation of Systems (QEST), Riverside, California, pp. 131–132 (2006)Google Scholar
  12. 12.
    Colón, M.A., Sankaranarayanan, S., Sipma, H.B.: Linear invariant generation using non-linear constraint solving. In: Hunt Jr., W.A., Somenzi, F. (eds.) CAV 2003. LNCS, vol. 2725, pp. 420–432. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Cousot, P., Halbwachs, N.: Automatic discovery of linear restraints among variables of a program. In: ACM Symposium on Principles of Programming Languages (POPL), Tucson, Arizona, pp. 84–96 (1978)Google Scholar
  14. 14.
    de Moura, L., Bjørner, N.S.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337–340. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  15. 15.
    Dijkstra, E.W.: Guarded commands, nondeterminacy and formal derivation of programs. Commun. ACM 18(8), 453–457 (1975)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Dimitrova, R., Ferrer Fioriti, L.M., Hermanns, H., Majumdar, R.: Probabilistic \({{\rm {CTL}}^{*}}\): the deductive way. In: Chechik, M., Raskin, J.-F. (eds.) TACAS 2016. LNCS, vol. 9636, pp. 280–296. Springer, Heidelberg (2016). doi: 10.1007/978-3-662-49674-9_16 CrossRefGoogle Scholar
  17. 17.
    Esparza, J., Gaiser, A., Kiefer, S.: Proving termination of probabilistic programs using patterns. In: Madhusudan, P., Seshia, S.A. (eds.) CAV 2012. LNCS, vol. 7358, pp. 123–138. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Ferrer Fioriti, L.M., Hermanns, H.: Probabilistic termination: soundness, completeness, and compositionality. In: ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), Mumbai, India, pp. 489–501 (2015)Google Scholar
  19. 19.
    Hart, S., Sharir, M., Pnueli, A.: Termination of probabilistic concurrent program. ACM Trans. Program. Lang. Syst. 5(3), 356–380 (1983)CrossRefzbMATHGoogle Scholar
  20. 20.
    Joyner, D., Čertík, O., Meurer, A., Granger, B.E.: Open source computer algebra systems: SymPy. ACM Commun. Comput. Algebra 45(3–4), 225–234 (2012)CrossRefzbMATHGoogle Scholar
  21. 21.
    Kaminski, B.L., Katoen, J.-P., Matheja, C., Olmedo, F.: Weakest precondition reasoning for expected run–times of probabilistic programs. In: Thiemann, P. (ed.) ESOP 2016. LNCS, vol. 9632, pp. 364–389. Springer, Heidelberg (2016). doi: 10.1007/978-3-662-49498-1_15 CrossRefGoogle Scholar
  22. 22.
    Katoen, J.-P., McIver, A.K., Meinicke, L.A., Morgan, C.C.: Linear-invariant generation for probabilistic programs: automated support for proof-based methods. In: Cousot, R., Martel, M. (eds.) SAS 2010. LNCS, vol. 6337, pp. 390–406. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  23. 23.
    Katoen, J., Zapreev, I.S., Hahn, E.M., Hermanns, H., Jansen, D.N.: The ins and outs of the probabilistic model checker MRMC. Perform. Eval. 68(2), 90–104 (2011)CrossRefGoogle Scholar
  24. 24.
    Kovács, L.: Reasoning algebraically about P-solvable loops. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 249–264. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Kozen, D.: Semantics of probabilistic programs. J. Comput. Syst. Sci. 22(3), 328–350 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  27. 27.
    McIver, A., Morgan, C.: Abstraction, Refinement and Proof for Probabilistic Systems. Monographs in Computer Science. Springer, New York (2005)zbMATHGoogle Scholar
  28. 28.
    Miné, A.: The octagon abstract domain. High.-Order Symb. Comput. 19(1), 31–100 (2006)CrossRefzbMATHGoogle Scholar
  29. 29.
    Monniaux, D.: An abstract analysis of the probabilistic termination of programs. In: Cousot, P. (ed.) SAS 2001. LNCS, vol. 2126, pp. 111–126. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  30. 30.
    Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995). ISBN 0-521-47465-5Google Scholar
  31. 31.
    Nielson, H.R.: A hoare-like proof system for analysing the computation time of programs. Sci. Comput. Program. 9(2), 107–136 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  32. 32.
    Rodríguez-Carbonell, E., Kapur, D.: Generating all polynomial invariants in simple loops. J. Symb. Comput. 42(4), 443–476 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Williams, D.: Probability with Martingales. Cambridge University Press, Cambridge (1991)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Gilles Barthe
    • 1
    Email author
  • Thomas Espitau
    • 2
  • Luis María Ferrer Fioriti
    • 3
  • Justin Hsu
    • 4
  1. 1.IMDEA Software InstituteMadridSpain
  2. 2.ENS CachanCachanFrance
  3. 3.Saarland UniversitySaarbrückenGermany
  4. 4.University of PennsylvaniaPhiladelphiaUSA

Personalised recommendations