Skip to main content
SpringerLink
Log in

Abstract Interpretation of Programs as Markov Decision Processes

  • Conference paper
  • First Online:
Static Analysis (SAS 2003)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2694))

Included in the following conference series:

Abstract

We propose a formal language for the specification of trace properties of probabilistic, nondeterministic transition systems, encompassing the properties expressible in Linear Time Logic. Those formulas are in general undecidable on infinite deterministic transition systems and thus on infinite Markov decision processes. This language has both a semantics in terms of sets of traces, as well as another semantics in terms of measurable functions; we give and prove theorems linking the two semantics. We then apply abstract interpretation-based techniques to give upper bounds on the worst-case probability of the studied property. We propose an enhancement of this technique when the state space is partitioned — for instance along the program points —, allowing the use of faster iteration methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Luca de Alfaro. Formal Verification of Probabilistic Systems. PhD thesis, Stanford University, Department of Computer Science, June 1998. CS-TR-98-1601.

    Google Scholar 

  2. Christel Baier, Edmund M. Clarke, Vasiliki Hartonas-Garmhausen, and Marta Kwiatkowska. Symbolic model checking for probabilistic processes. In P. Degano, R. Gorrieri, and A. Marchetti-Spaccamela, editors, Automata, Languages and Programming (ICALP’97), volume 1256 of LNCS. Springer, 1997.

    Google Scholar 

  3. A. Bianco and L. de Alfaro. Model checking of probabilistic and nondeterministic systems. In FST TCS’95: Foundations of Software Technology and Theoretical Computer Science, volume 1026 of Lecture Notes in Computer Science, pages 499–513. Springer-Verlag, 1995.

    Google Scholar 

  4. C. Baier, M. Kwiatkowska, and G. Norman. Computing probability bounds for linear time formulas over concurrent probabilistic systems. Electronic Notes in Theoretical Computer Science, 21, 1999.

    Google Scholar 

  5. Edmund M. Clarke, Jr., Orna Grumberg, and Doron A. Peled. Model Checking. MIT Press, 1999.

    Google Scholar 

  6. Rance Cleaveland, Scott A. Smolka, and Amy E. Zwarico. Testing preorders for probabilistic processes. In Werner Kuich, editor, Automata, Languages and Programming, 19th International Colloquium, volume 623 of Lecture Notes in Computer Science, pages 708–719, Vienna, Austria, 13–17 July 1992. Springer-Verlag.

    Google Scholar 

  7. C. Courcoubetis and M. Yannakakis. Markov decision processes and regular events. In Proc. ICALP’90, volume 443 of LNCS, pages 336–349. Springer, 1990.

    Google Scholar 

  8. Patrick Cousot. Méthodes itératives de construction et d’approximation de points fixes d’opérateurs monotones sur un treillis, analyse sémantique de programmes. ThËse d’état Ës sciences mathématiques, Université scientifique et médicale de Grenoble, Grenoble, France, 21 mars 1978.

    Google Scholar 

  9. Patrick Cousot and Radhia Cousot. Abstract interpretation and application to logic programs. J. Logic Prog., 2–3(13):103–179, 1992.

    Article  MathSciNet  Google Scholar 

  10. L. de Alfaro and R. Majumdar. Quantitative solution of omega-regular games. In STOC’01, 33rd Annual ACM Symposium on Theory of Computing. ACM, 2001.

    Google Scholar 

  11. Hans Hansson and Bengt Jonsson. A logic for reasoning about time and reability. Technical Report R90-13, Swedish Institute of Computer Science, December 1990.

    Google Scholar 

  12. Marta Z. Kwiatkowska, Gethin Norman, Roberto Segala, and Jeremy Sproston. Verifying quantitative properties of continuous probabilistic timed automata. Technical Report CSR-00-6, University of Birmingham, School of Computer Science, March 2000.

    Google Scholar 

  13. Marta Z. Kwiatkowska, Gethin Norman, Roberto Segala, and Jeremy Sproston. Verifying quantitative properties of continuous probabilistic timed automata. In C. Palamidessi, editor, CONCUR 2000-Concurrency Theory 11th International Conference, number 1877 in LNCS. Springer, 2000.

    Chapter  Google Scholar 

  14. Michael L. Littman, Anthony R. Cassandra, and Leslie Pack Kaelbling. Efficient dynamic-programming updates in partially observable markov decision processes. Technical Report CS-95-19, Brown University, 1995.

    Google Scholar 

  15. A. McIver. Reasoning about efficiency within a probabilistic μ-calculus. In Proc. of PROBMIV, pages 45–58, 1998. Technical Report CSR-98-4, University of Birmingham, School of Computer Science.

    Google Scholar 

  16. David Monniaux. Abstract interpretation of probabilistic semantics. In Seventh International Static Analysis Symposium (SAS’00), number 1824 in Lecture Notes in Computer Science, pages 322–339. Springer Verlag, 2000. Extended version on the author’s web site.

    Google Scholar 

  17. David Monniaux. An abstract analysis of the probabilistic termination of programs. In 8th International Static Analysis Symposium (SAS’01), number 2126 in Lecture Notes in Computer Science, pages 111–126. Springer Verlag, 2001.

    Google Scholar 

  18. David Monniaux. An abstract Monte-Carlo method for the analysis of probabilistic programs (extended abstract). In 28th Symposium on Principles of Programming Languages (POPL’01), pages 93–101. Association for Computer Machinery, 2001.

    Google Scholar 

  19. David Monniaux. Backwards abstract interpretation of probabilistic programs. In European Symposium on Programming Languages and Systems (ESOP’01), number 2028 in Lecture Notes in Computer Science, pages 367–382. Springer Verlag, 2001.

    Google Scholar 

  20. David Monniaux. Abstraction of expectation functions using gaussian distributions. In Lenore D. Zuck, Paul C. Attie, Agostino Cortesi, and Supratik Mukhopadhyay, editors, Verification, Model Checking, and Abstract Interpretation: VMCAI’03, number 2575 in Lecture Notes in Computer Science, pages 161–173. Springer Verlag, 2003.

    Chapter  Google Scholar 

  21. Jacques Neveu. Mathematical Foundations of the Calculus of Probabilities. Holden-Day, 1965.

    Google Scholar 

  22. Jacques Neveu. Bases mathématiques du calcul des probabilités. Masson et Cie, Éditeurs, Paris, 1970. Préface de R. Fortet. Deuxième édition, revue et corrigée.

    MATH  Google Scholar 

  23. Martin L. Puterman. Markov decision processes: Discrete stochastic dynamic programming. Wiley series in probability and mathematical statistics. John Wiley & Sons, 1994.

    Google Scholar 

  24. Roberto Segala. Modeling and Verification of Randomized Distributed Real-Time Systems. PhD thesis, Massachusetts Institute of Technology, 1995. Technical report MIT/LCS/TR-676.

    Google Scholar 

  25. W. Thomas. Automata on infinite objects. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science, vol. B, pages 135–191. Elsevier, 1990.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Département d’Informatique, École Normale Supérieure, 45, rue d’Ulm, 75230, Paris cedex 5, France

    David Monniaux

Authors
  1. David Monniaux
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. École Polytechnique, STIX, 91128, Palaiseau cedex, France

    Radhia Cousot

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Monniaux, D. (2003). Abstract Interpretation of Programs as Markov Decision Processes. In: Cousot, R. (eds) Static Analysis. SAS 2003. Lecture Notes in Computer Science, vol 2694. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44898-5_13

Download citation

  • DOI: https://doi.org/10.1007/3-540-44898-5_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40325-8

  • Online ISBN: 978-3-540-44898-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation