Abstract
We develop a symbolic, logic-based technique for constructing optimal control policies in some transition systems where state spaces are large or infinite. These systems are presented as iterations of finite sets of guarded assignments which have costs. The optimality objective is to minimize the total costs of system executions reaching the set characterized by a given target predicate. Guards are predicates and control policies are expressed by tuples of guards. The optimal control policy refines the control policy of the given system. It is generated from the target predicate by an iteration based on backwards induction. This iterative procedure amounts to a variant of the symbolic algorithm generating the reachability precondition; the latter characterizes the states from which some system execution reaches the target set. The main difference is the introduction of greedy and cost-dependent iteration steps.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abrial, J.-R.: Modeling in Event-B: System and Software Engineering. Cambridge Univ. Press, Cambridge (to be published, 2008)
Akin, E.: The General Topology of Dynamical Systems. Amer. Math. Soc., Providence (1998)
Back, R.-J., von Wright, J.: Refinement Calculus. Springer, New York (1998)
Başa, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. SIAM, Philadelphia (1999)
Bellman, R.: Dynamic Programming. Princeton Univ. Press, Princeton (1957)
Bertsekas, D.: Dynamic Programming and Optimal Control. Athena Scientific, Belmont (2000)
Boutilier, C., Reiter, R., Price, B.: Symbolic dynamic programming for first-order MDPs. In: Proc. 7th Int. Joint Conf. Artificial Intelligence, pp. 690–697. M. Kaufmann, San Francisco (2001)
Bouyer, P., Cassez, F., Fleury, E., Larsen, K.: Synthesis of optimal strategies using HyTech. Electronic Notes Theor. Computer Sci. 119, 11–31 (2005)
Bouyer, P., Brihaye, T., Chevalier, M.: Weighted o-minimal hybrid systems are more decidable than weighted timed automata! In: Artemov, S.N., Nerode, A. (eds.) LFCS 2007. LNCS, vol. 4514, pp. 69–83. Springer, Heidelberg (2007)
Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. MIT Press, Cambridge (1999)
de Bakker, J.W., de Roever, W.P.: A calculus for recursive program schemes. In: Nivat, M. (ed.) Proc. 1st Int. Conf. Automata, Languages and Programming, pp. 167–196. North-Holland, Amsterdam (1973)
Dijkstra, E.W.: A note on two problems in connexion with graphs. Numerische Mathematik 1, 269–271 (1959)
Dijkstra, E.W.: A Discipline of Programming. Prentice-Hall, Englewood Cliffs (1976)
Floyd, R.: Algorithm 97 (Shortest path). Commun. ACM 5, 345 (1962)
Floyd, R.: Assigning meanings to programs. In: Schwartz, J.T. (ed.) Proc. Symp. Appl. Mathematics, vol. 19, pp. 19–31. Amer. Math. Soc., Providence (1967)
Henzinger, T.A., Majumdar, R., Raskin, J.-F.: A classification of symbolic transition systems. ACM Trans. Computational Logic 6, 1–32 (2005)
Kupferman, O., Vardi, M.Y.: An Automata-theoretic Approach to Reasoning about Infinite-state Systems. In: Proc. 12th Int. Conf. Computer Aided Verification. LNCS, vol. 1855, pp. 36–52. Springer, Berlin (2006)
Lind, D., Marcus, B.: An Introduction to Symbolic Dynamics and Coding. Cambridge Univ. Press, Cambridge (1995)
Schmidt, D.A.: Structure-preserving binary relations for program abstraction. In: Mogensen, T.Æ., Schmidt, D.A., Sudborough, I.H. (eds.) The Essence of Computation. LNCS, vol. 2566, pp. 245–265. Springer, Heidelberg (2002)
Sintzoff, M.: Iterative Synthesis of Control Guards Ensuring Invariance and Inevitability in Discrete-decision Games. In: Owe, O., Krogdahl, S., Lyche, T. (eds.) From Object-Orientation to Formal Methods. LNCS, vol. 2635, pp. 272–301. Springer, Heidelberg (2004)
Sontag, E.D.: Mathematical Control Theory. Springer, New-York (1990)
van Lamsweerde, A., Sintzoff, M.: Formal derivation of strongly concurrent programs. Acta Informatica 12, 1–31 (1979)
Vinter, R.: Optimal Control. Birkhäuser, Boston (2000)
Warshall, S.: A theorem on boolean matrices. J. ACM 9, 11–12 (1962)
Wiggins, S.: Introduction to Applied Nonlinear Dynamical Systems and Chaos. Springer, New York (1990)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sintzoff, M. (2008). Synthesis of Optimal Control Policies for Some Infinite-State Transition Systems. In: Audebaud, P., Paulin-Mohring, C. (eds) Mathematics of Program Construction. MPC 2008. Lecture Notes in Computer Science, vol 5133. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70594-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-540-70594-9_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70593-2
Online ISBN: 978-3-540-70594-9
eBook Packages: Computer ScienceComputer Science (R0)