Synthesis of Optimal Strategies for Differential Games by Neural Networks
The paper deals with the numerical approximation of optimal strategies for two-person zero-sum differential games of pursuit-evasion type by neural networks. Thereby, the feedback strategies can be computed in real-time after the training of appropriate neural networks. For this purpose, sufficiently many optimal trajectories and their associated open-loop representations of the optimal feedback strategies must be computed, to provide data for training and cross-validation of the neural networks. All the precomputations can be carried through in a highly parallel way. This approach turns out to be applicable for differential games of more general type.
The method is demonstrated for a modified cornered rat game where a pursuing cat and an evading rat, both moving in simple motion, are constrained to a rectangular arena. Two holes in the walls surrounding the arena enable the rat to evade once and for all, if the rat is not too far from these holes. The optimal trajectories in the escape zone can be computed analytically. In the capture zone, a game of degree is employed with terminal time as payoff. To compute optimal trajectories for this secondary game, the time evolution of the survival region for the rat is determined via a sequence of discretized games.
The combination of these methods permits the computation of more than a thousand trajectories leading to some ten thousand sample patterns which relate the state variables to the values of the optimal strategies. These data exhibit characteristic properties of the optimal strategies. It is shown that these properties can be extracted from the data by use of neural networks. By means of the trained networks, about 200 trajectories arc finally simulated. The pursuer as well as the evader acts according to the controls proposed by the neural networks. Despite the simple structure of the neural networks used in this study, the strategics based upon them show a reasonable, close to optimal performance in a large variety of simulations of the pursuit-evasion game under consideration.
KeywordsDifferential games neural networks real-time computation pursuit-evasion games optimal strategies numerical solution of differential games cornered rat game
Unable to display preview. Download preview PDF.
- Ba§ar, T., and Olsder, G. J., Dynamic Noncooperative Game Theory, Academic Press, London, 1982, 2nd ed., 1994.Google Scholar
- Benveniste, A., Metivier, M., and Priouret, P., Algorithmes adaptatifs et approximations stochastique, Masson, Paris, 1987.Google Scholar
- Breakwell, J. V., Zero-Sum Differential Games with Terminal Payoff, in: P. Hagedorn, H. W. Knobloch, and G. J. Olsder (Eds.), Differential Games and Applications, Lecture Notes in Control and Information Sciences 3, Springer, Berlin, 1977, 70 – 95.Google Scholar
- Breitner, M. H., Construction of the Optimal Feedback Controller for Constrained Optimal Control Problems with Unknown Disturbances, in: R. Bulirsch, and D. Kraft (Eds.), Computational Optimal Control, International Series of Numerical Mathematics 115, Birkhauser, Basel, 1994, 147 – 162.Google Scholar
- Breitner, M. H., Real-Time Capable Approximation of Optimal Strategies in Complex Differential Games, in: M. Breton, G. Zaccour (Eds.), Proceedings of the Sixth International Symposium on Dynamic Games and Applications, St-Jovite, Quebec, GERAD, Ecole des Hautes Etudes Commerciales, Montreal, 1994, 370 – 374.Google Scholar
- Breitner, M. H., and Pesch H. J., Reentry Trajectory Optimization under Atmospheric Uncertainty as a Differential Game, in: T. Ba§ar, and A. Haurie (Eds.), Advances in Dynamic Games and Applications, Annals of the International Society of Dynamic Games 1, Birkhauser, Boston, 1994, 70 – 88.Google Scholar
- Breitner, M. H., Pesch, H. J., and Grimm, W., Complex Differential Games of Pursuit-Evasion Type with State Constraints, Part 1: Necessary Conditions for Optimal Open-Loop Strategies, Part 2: Numerical Computation of Optimal Open-Loop Strategies, Journal of Optimization Theory and Applications 78, 1993, 419–441, 443 – 463.MathSciNetCrossRefGoogle Scholar
- Bryson, A. E., and Ho, Y.-C., Applied Optimal Control, Hemisphere, New York, 1975.Google Scholar
- Gabler, I., Numerische Berechnung optimaler Strategien eines Diffe-rentialspiels und Approximation durch Neuronale Netze, Diploma thesis, Department of Mathematics, Munich University of Technology, Munich, 1993.Google Scholar
- Gabler, I., Miesbach, S., Breitner, M. H., and Pesch, H. J., Synthesis of Optimal Strategies for Differential Games by Neural Networks, Report No. 468, Deutsche Forschungsgemeinschaft, Schwer-punkt “Anwendungsbezogene Optimierung und Steuerung”, Department of Mathematics, Munich University of Technology, Munich, 1993.Google Scholar
- Hertz, J., Krogh, A., and Palmer, R. G., Introduction to the Theory of Neural Computation, Addison-Wesley, Redwood City, 1991.Google Scholar
- Isaacs, R., Differential Games, John Wiley & Sons, New York, 1965, Krieger, New York, 1975.Google Scholar
- Jarmark, B., and Bengtsson, H., Near-Optimal Flight Trajectories Generated by Neural Networks, Internal Report, Saab/Scania Aircraft Division, Linkoping, 1989.Google Scholar
- Koslik, B., Pesch, H. J., Breitner, M. H., and von Stryk, O., Optimal Control of a Complex Concern Model by Direct and Indirect Optimization Methods, to appear in: H. Engl, and H. Neunzert (Eds.), Proceedings of the 8th Conference of the European Consortium for Mathematics in Industry, ECMI Series, Teubner, Stuttgart, 1995.Google Scholar
- Lachner, R., Breitner, M. H., and Pesch H. J., Three-Dimensional Air Combat: Numerical Solution of Complex Differential Games, in: G. J. Olsder (Ed.), New Trends in Dynamic Games and Applications, this volume, Birkhauser, Boston, 1995, 165 – 190.Google Scholar
- Lachner, R., Breitner, M.H., and Pesch H.J., Optimal Strategies of a Complex Pursuit-Evasion Game, to appear in Journal of Computing and Information, 1995.Google Scholar
- Miesbach, S., Bahnfuhrung von Robotern mit Neuronalen Netzen, PhD thesis, Department of Mathematics, Munich University of Technology, Munich, 1995.Google Scholar
- Miesbach, S., and Schurmann, B., Wenn Roboter arbeiten lernen: Ideen und Methoden der Neuroinformatik zur Regelung und Steuerung, Informationstechnik 33, 1991, 300 – 309.Google Scholar
- Miller, W. T., Sutton, R. S., and Werbos, P. J. (Eds.), Neural Networks for Control, MIT Press, Cambridge, 1990.Google Scholar
- Nguyen, D., and Widrow, B., The Truck Backer-Upper: An Example of Self-Learning in Neural Networks, in: W. T. Miller, R. S. Sutton, and P. J. Werbos (Eds.), Neural Networks for Control, The MIT Press, Cambridge, 1990, 287 – 299.Google Scholar
- Pesch, H. J., Offline and Online Computation of Optimal Trajectories in the Aerospace Field, in: A. Miele and A. Salvetti (Eds.), Applied Mathematics in Aerospace Science and Engineering, Plenum, New York, 1994, 165–219.Google Scholar
- Pesch, H. J., Solving Optimal Control and Pursuit-Evasion Game Problems of High Complexity, in: R. Bulirsch, and D. Kraft (Eds.), Computational Optimal Control, International Series of Numerical Mathematics 115, Birkhauser, Basel, 1994, 43–64.Google Scholar
- Proceedings of the NIPS Neural Information Processing Systems, Morgan Kaufmann, San Mateo, CA, annually since 1987.Google Scholar
- Ritter, H., Martinetz, T., and Schulten, K., Neuronale Netze: Eine Einfiihrung in die Neuroinformatik selbstorganisierender Netz-werke, Addison-Wesley, Bonn, 1991.Google Scholar
- Rojas, R., Theorie der neuronalen Netze, Springer, Berlin, 1993.Google Scholar
- von Stryk, O., Numerical Solution of Optimal Control Problems by Direct Collocation, in: R. Bulirsch, A. Miele, J. Stoer, and K.-H. Well (Eds.), Optimal Control, International Series of Numerical Mathematics 111, Birkhauser, Basel, 1993, 129 – 143.Google Scholar