Abstract
Policy iterations is a technique based on game theory that relies on a sequence of numerical optimization queries to compute the fixpoint of a set of equations. It has been proposed to support the static analysis of programs as an alternative to widening, when the latter is ineffective. This happens for instance with highly numerical codes, such as found at cores of control command applications. In this paper we present a complete, yet practical, description of the use of policy iteration in this context. We recall the rationale behind policy iteration and address required steps towards an automatic use of it: synthesis of numerical templates, floating point semantics of the analyzed program and issues with the accuracy of numerical solvers.
Similar content being viewed by others
Notes
The term strategy is also used in the literature, with equivalent meaning.
Although the \(k\)-inductive invariants can be made (1)-inductive by adding extra variables, representing past values of program variables, in their expression.
Like the one in Fig. 1.
All figures are rounded to the fourth digit.
Although the minimum volume (Löwner-Johns) ellipsoid [6], Section 8.4] could be a reasonable choice.
\(\vee \) is often used instead in the policy iteration literature.
Or a large enough guess can be used. Thanks to the fast convergence of min-policy iterations, there is often no need for this postfixpoint to be close from the fixpoint eventually computed.
More precisely, first determine which \(b_{i,j}\) are \(\pm \infty \) in the least fixpoint in \({\overline{\mathbb {R}}}^{np}\) greater than \(b_i\), then compute a greatest fixpoint for the remaining values in \({\mathbb {R}}\).
More precisely, for a given policy \(\overline{F}_{i+1}\), once determined which \(b_{i,j}\) are \(\pm \infty \) there is a unique greatest fixpoint for the remaining \(b_{i,j} \in {\mathbb {R}}\), hence finitely many possible \(b_{i+1}\).
There is usually no best ellipsoidal invariant, so we have to resort on a heuristic.
Moreover, \(x^2\) being an homogeneous degree two polynomial is easier to express in semi-definite programs than linear constraints which would require an extra dimension to encode linear terms.
Although they are clearly the maximal values of each template under constraint \(0 \le x_1 \le 1 \wedge 0 \le x_2 \le 1\).
For, denoting \(p\) the previous degree two polynomial, \(\displaystyle \lim _{x \rightarrow \infty } p(x) = -\infty \), whatever the values of \(\lambda _1\) and \(\lambda _2\).
Only one constraint here for ease of exposition. Everything works the same with multiple constraints.
Thanks to Timothy Wang for pointing this to us.
Although, in our case, this positive definiteness check only accounts for a very small part of the total analysis time. Thus, the eventual overhead would remain limited.
Usual implementation of type double in C.
Order of evaluation matters since floating point addition is not associative.
A similar proof can be performed if the sum is not computed in this left-right order.
The relative difference between the \(b_{v',j}\) and the \(b'_{v',j}\) or the \(c\) and \(c'\) never exceeded \(10^{-10}\) in our experiments (to be compared to the \(10^{-4}\) padding previously applied).
References
Adjé A, Gaubert S, Goubault E (2010) Coupling policy iteration with semi-definite relaxation to compute accurate numerical invariants in static analysis. In: ESOP, pp 23–42
Alegre F, Féron E, Pande S (2009) Using ellipsoidal domains to analyze control systems software. arXiv:0909.1977
Boldo S, Melquiond G (2011) Flocq: a unified library for proving floating-point algorithms in Coq. In: Proceedings of the 20th IEEE symposium on computer arithmetic. Tübingen, pp 243–252
Bouissou O, Seladji Y, Chapoutot A (2012) Acceleration of the abstract fixpoint computation in numerical program analysis. J Symb Comput 47(12):1479–1511
Boyd S, El Ghaoui L, Féron E, Balakrishnan V (1994) Linear matrix inequalities in system and control theory, volume 15 of SIAM. SIAM, Philadelphia
Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, Cambridge
Champion A, Delmas R, Dierkes M, Garoche P-L, Jobredeaux R, Roux P (2013) Formal methods for the analysis of critical control systems models: combining non-linear and linear analyses. In: Charles P, Michael D, (eds), Formal methods for industrial critical systems—18th international workshop, FMICS 2013, Madrid, Spain, September 23–24, 2013. Proceedings, volume 8187 of Lecture Notes in Computer Science, pp 1–16. Springer
Costan A, Gaubert S, Goubault E, Martel M, Putot S (2005) A policy iteration algorithm for computing fixed points in static analysis of programs. In: CAV, pp 462–475
Cousot P, Cousot R (1977) Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: POPL, pp 238–252
Cousot P, Cousot R (1979) Systematic design of program analysis frameworks. In: POPL, pp 269–282
Cousot P, Cousot R (1992) Abstract interpretation frameworks. J Log Comput 2(4):511–547
Cousot P, Halbwachs N (1978) Automatic discovery of linear restraints among variables of a program. In: POPL, pp 84–96
Feautrier P, Gonnord L (2010) Accelerated invariant generation for c programs with aspic and c2fsm. Electron Notes Theor Comput Sci 267(2):3–13
Feret J (2004) Static analysis of digital filters. In: ESOP, number 2986 in LNCS. Springer
Feret J (2005) Numerical abstract domains for digital filters. In: International workshop on Numerical and Symbolic Abstract Domains (NSAD)
Féron E (2010) From control systems to control software. IEEE Control Syst 30(6):50–71
Gaubert S, Goubault E, Taly A, Zennou S (2007) Static analysis by policy iteration on relational domains. In: ESOP, pp 237–252
Gawlitza T, Seidl H (2007) Precise fixpoint computation through strategy iteration. In: ESOP, pp 300–315
Gawlitza TM, Seidl H (2010) Computing relaxed abstract semantics w.r.t. quadratic zones precisely. In: SAS, pp 271–286
Gawlitza TM, Seidl H, Adjé A, Gaubert S, Goubault E (2012) Abstract interpretation meets convex optimization. J Symb Comput 47(12):1416–1446
Ghorbal K, Goubault E, Putot S (2009) The zonotope abstract domain taylor1+. In: CAV, pp 627–633
Gopan D, Reps TW (2006) Lookahead widening. In: CAV, pp 452–466
Goubault E, Putot S (2011) Static analysis of finite precision computations. In: VMCAI, pp 232–247
Haddad WM, Chellaboina VS (2008) Nonlinear dynamical systems and control: a lyapunov-based approach. Princeton University Press, Princeton
Halbwachs N, Henry J (2012) When the decreasing sequence fails. In: SAS, pp 198–213
Halbwachs Nicolas, Proy Yann-Erick, Roumanoff Patrick (1997) Verification of real-time systems using linear relation analysis. Formal Methods in System Design 11(2):157–185
Higham NJ (1996) Accuracy and stability of numerical algorithms. Society for Industrial and Applied Mathematics, Philadelphia
IEEE Computer Society (2008) IEEE standard for floating-point arithmetic. In: IEEE Standard 754–2008
Lyapunov AM (1947) Problème général de la stabilité du mouvement. Annals of Mathematics Studies 17. Princeton University Press, Princeton
Miné A (2001) The octagon abstract domain. In: AST 2001 in WCRE 2001, IEEE, pp 310–319. IEEE CS Press
Miné A (2004) Relational abstract domains for the detection of floating-point run-time errors. In: ESOP, volume 2986 of LNCS, pp 3–17. Springer, http://www.di.ens.fr/~mine/publi/article-mine-esop04.pdf
Monniaux D (2005) Compositional analysis of floating-point linear numerical filters. In: CAV, pp 199–212
Roozbehani M, Féron E, Megretski A (2005) Modeling, optimization and computation for software verification. In: HSCC, pp 606–622
Roux P (2013) Static analysis of control command systems: synthetizing non linear invariants. PhD thesis, Institut Supérieur de l’Aéronautique et de l’Espace
Roux P, Garoche P-L (2013) Integrating policy iterations in abstract interpreters. In: Dang Van Hung and Mizuhito Ogawa, (eds), Automated technology for verification and analysis—11th international symposium, ATVA 2013, Hanoi, Vietnam, October 15–18, 2013. Proceedings, volume 8172 of Lecture Notes in Computer Science, pp 240–254. Springer
Roux P, Garoche P-L (2014) Computing quadratic invariants with min- and max-policy iterations: a practical comparison. In: Jones CB, Pihlajasaari P, Sun J (eds), FM 2014: formal methods—19th international symposium, Singapore, May 12–16, 2014. Proceedings, volume 8442 of Lecture Notes in Computer Science, pp 563–578. Springer
Roux P, Jobredeaux R, Garoche P-L, Féron E (2012) A generic ellipsoid abstract domain for linear time invariant systems. In: HSCC, pp 105–114
Rump SM (2006) Verification of positive definiteness. BIT Numer Math 46:433–452
Rump SM (2010) Verification methods: Rigorous results using floating-point arithmetic. Acta Numer 19:287–449
Schrammel P, Jeannet B (2011) Logico-numerical abstract acceleration and application to the verification of data-flow programs. In: SAS, pp 233–248
Seladji Y, Bouissou O (2013) Numerical abstract domain using support functions. NFM 7871:155–169
The Coq Development Team (2013) The Coq proof assistant reference manual, 2012. Version 8.4. Springer, Heidelberg
Vandenberghe L, Boyd S (1996) Semidefinite programming. SIAM Rev 38(1):49–95
Acknowledgments
We would like to deeply thank the anonymous reviewers for their highly relevant comments to improve this paper. This work has been partially supported by the ANR-INSE-2012-007 Grant CAFEIN and the Aerospace Valley competitivity cluster.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Roux, P., Garoche, PL. Practical policy iterations. Form Methods Syst Des 46, 163–196 (2015). https://doi.org/10.1007/s10703-015-0230-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10703-015-0230-7