Optimization Techniques for State-Constrained Control and Obstacle Problems

  • A. B. Kurzhanski
  • I. M. Mitchell
  • P. Varaiya


The design of control laws for systems subject to complex state constraints still presents a significant challenge. This paper explores a dynamic programming approach to a specific class of such problems, that of reachability under state constraints. The problems are formulated in terms of nonstandard minmax and maxmin cost functionals, and the corresponding value functions are given in terms of Hamilton-Jacobi-Bellman (HJB) equations or variational inequalities. The solution of these relations is complicated in general; however, for linear systems, the value functions may be described also in terms of duality relations of convex analysis and minmax theory. Consequently, solution techniques specific to systems with a linear structure may be designed independently of HJB theory. These techniques are illustrated through two examples.


Nonlinear systems control synthesis state constraints obstacle problems dynamic programming variational inequalities convex analysis 


  1. 1.
    KURZHANSKI, A. B., and VARAIYA, P., Optimization Techniques for Reachability Analysis, Journal of Optimization Theory and Applications, Vol. 108, pp. 227–251, 2001.MathSciNetCrossRefGoogle Scholar
  2. 2.
    KURZHANSKI, A. B., and VARAIYA, P., Reachability under State Constraints. The Ellipsoidal Technique, Proceedings of the IFAC 2002 World Congress, Barcelona, Spain, Elsevier, Amsterdem, Holland, 2002.Google Scholar
  3. 3.
    KURZHANSKI, A. B., and VARAIYA, P., On Some Nonstandard Dynamic Programming Problems of Control Theory, Variational Methods and Applications, Edited by F. Giannessi and A. Maugeri, Kluwer Academic Publishers, New York, NY, pp. 613–627, 2004.Google Scholar
  4. 4.
    GUSEV, M. I., and KURZHANSKI, A. B., Optimization of Controlled Systems with Bounds on the Controls and State Coordinates, Differential Equations, Vol. 7, pp. 1591–1602, 1971, and Vol. 7, pp. 1789–1800, 1971.Google Scholar
  5. 5.
    KRASOVSKII N. N., Game-Theoretic Problems on the Encounter of Motions, Nauka, Moscow, Russia, 1970 (in Russian). English Translation: Rendezvous Game Problems, National Technical Information Service, Springfield, Virginia, 1971.Google Scholar
  6. 6.
    KURZHANSKI, A. B., Control and Observation under Uncertainty, Nauka, Moscow, Russia, 1977 (in Russian).Google Scholar
  7. 7.
    KURZHANSKI, A. B., and OSIPOV, YU. S., On Optimal Control under Restricted Coordinates, PMM (Applied Mathematics and Mechanics), Vol. 33, pp. xxx–xxx, 1969.Google Scholar
  8. 8.
    LEITMANN, G., Optimality and Reachability via Feedback Controls, Dynamic Systems and Microphysics, Edited by A. Blaquière and G. Leitmann, Academic Press, New York, NY, pp. 119–141, 1982.Google Scholar
  9. 9.
    LYGEROS, J., On the Relation of Reachability to Minimum-Cost Control, Proceedings of the 41st IEEE Control and Decision Conference, Las Vegas, Nevade, 2002.Google Scholar
  10. 10.
    KURZHANSKI, A. B., MITCHELL, I. M., and VARAIYA, P., Control Synthesis for State Constrained and Obstacle Problems, Proceedings of the IFAC Symposium NOLCOS-2004, Stuttgart, Germany, Elsevier, Amsterdam, Holland, 2004.Google Scholar
  11. 11.
    FILIPPOV A. F., Differential Equations with a Discontinuous Right-Hand Side, Kluwer Academic Publishers, Dondrecht, Holland, 1988.Google Scholar
  12. 12.
    AUBIN J. P., Viability Theory, Birkhauser, Boston, Massachusetts, 1991.zbMATHGoogle Scholar
  13. 13.
    ROCKAFELLAR R. T., Convex Analysis, 2nd Edition, Princeton University Press, Princeton, New Jersey, 1999.zbMATHGoogle Scholar
  14. 14.
    LYGEROS, J., TOMLIN, C., and SASTRY, S., Controllers for Reachability Specifications for Hybrid Systems, Automatica, Vol. 35, pp. 349–370, 1999.zbMATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    FLEMING, W. H., and SONER, H. M., Controlled Markov Processes and Viscosity Solutions, Springer Verlag, New York, NY, 1993.zbMATHGoogle Scholar
  16. 16.
    CRANDALL, M. G., EVANS, L. C., and LIONS P. L., Some Properties of Solutions of Hamilton-Jacobi Equations, Transactions of the American Mathematical Society, Vol. 282, pp. 487–502, 1984.zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    LIONS P. L., Viscosity Solutions and Optimal Control, Proceedings of the International Congress on industrial and Applied Mathematics, ICIAM 91, SIAM Publications, Philadelphia, Pennsylvania, pp. 182–195, 1992.Google Scholar
  18. 18.
    SUBBOTIN, A. I., Generalized Solutions of First-Order PDE's: The Dynamic Optimization Perspective, Birkhauser, Boston, Massachusettss, 1995.Google Scholar
  19. 19.
    BARDI, M., and CAPUZZO-DOLCETTA, I., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Birkhauser, Boston, Massachusetts, 1997.zbMATHGoogle Scholar
  20. 20.
    SETHIAN, J. A., Level Set Methods and Fast Marching Methods, Cambridge University Press, Cambridge, UK, 1999.zbMATHGoogle Scholar
  21. 21.
    OSHER, S., and FEDKIW, R., Level Set Methods and Implicit Surfaces, Springer Verlag, New York, NY, 2002.Google Scholar
  22. 22.
    MITCHELL, I., A Toolbox of Level Set Methods, Technical Report TR-2004-09, Department of Computer Science, University of British Columbia, Vancouver, BC, Canada, 2004 (Available at
  23. 23.
    VARAIYA P., Reach Set Computation Using Optimal Control, Proceedings of the KIT Workshop on Verification of Hybrid Systems, Verimag, Grenoble, France, 1998.Google Scholar
  24. 24.
    ROCKAFELLAR, R. T., and WETS, R. J. B., Variational Analysis, Springer Verlag, New York, NY, 1998.zbMATHGoogle Scholar
  25. 25.
    FAN, K., Minimax Theorems, Proceedings of the National Academy of Sciences of the USA, Vol. 31, pp. 42–47, 1953.Google Scholar
  26. 26.
    DEMYANOV, V. F., and MALOZEMOV, X. X., Introduction to the Minmax, Halsted Press, New York, NY, 1974.Google Scholar
  27. 27.
    BOYD, S., EL GHAOUI, L., FERON, E., and BALAKRISHNAN, V., Linear Matrix Inequalities in System and Control Theory, SIAM Publications, Philadelphia, Pennsylvania, 1994.zbMATHGoogle Scholar
  28. 28.
    KURZHANSKI, A. B., MITCHELL, I. M., and VARAIYA, P., Optimization Techniques for State-Constrained and Obstacle Problems, Available at (

Copyright information

© Springer Science+Business Media, Inc. 2006

Authors and Affiliations

  • A. B. Kurzhanski
    • 1
  • I. M. Mitchell
    • 2
  • P. Varaiya
    • 3
  1. 1.Department of Computational Mathematics and CyberneticsMoscow State (Lomonosov) UniversityMoscowRussia
  2. 2.Department of Computer ScienceUniversity of British ColumbiaVancouverCanada
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of California at BerkeleyBerkeleyUSA

Personalised recommendations