The Extended Ritz Method in Stochastic Functional Optimization: An example of Dynamic Routing in Traffic Networks

  • M. Baglietto
  • M. Sanguineti
  • R. Zoppoli
Part of the Applied Optimization book series (APOP, volume 82)


The classical Ritz method constrains the admissible solutions of functional optimization problems to take on the structure of linear combinations of fixed basis functions. Under general assumptions, the coefficients of such linear combinations become the unknowns of a finite-dimensional nonlinear programming problem. We propose to insert ”free” parameters to be optimized in the basis functions, too. This justifies the term “Extended Ritz Method.” If the optimal solutions of functional optimization problems belong to classes of d-variable functions characterized by suitable regularity properties, the Extended Ritz Method may outperform the Ritz method in that the number of free parameters increases moderately (e.g., polynomially) with d, whereas the latter method may be ruled out by the curse of dimensionality. Once the functional optimization problem has been approximated by a nonlinear programming one, the solution of the latter problem is obtained by stochastic approximation techniques. The overall procedure turns out to be effective in high-dimensional settings, possibly in the presence of several decision makers. In such a context, we focus our attention on large-scale traffic networks such as communication networks, freeway systems, etc. Traffic flows may vary over time. Then the nodes of the networks (i.e., the decision makers acting at the nodes) may be requested to modify the traffic flows to be sent to their neighboring nodes. Consequently, a dynamic routing problem arises that cannot be solved analytically. In particular, we address store-and-forward packet switching networks, which exhibit the essential difficulties of traffic networks. Simulations performed on complex communication networks show the effectiveness of the proposed method.


Functional optimization curse of dimensionality polynomially complex approximating networks Ritz method stochastic approximation team optimal control traffic and communication networks 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sjöberg, J., Zhang, Q., Ljung, L., Benveniste, A., Glorennec, P.-Y., Delyon, B., Hjalmarsson, H., and Juditsky, A., Nonlinear Black-Box Modeling in System Identification: A Unified Overview, Automatica, Vol. 31, pp. 1691–1724, 1995.zbMATHCrossRefGoogle Scholar
  2. 2.
    Leshno, M., LIN, V. YA., Pinkus, A., and Schocken, S., Multilayer Feed-forward Networks with a Nonpolynomial Activation Function Can Approximate Any Function, Neural Networks, Vol. 6, pp. 861–867, 1993.CrossRefGoogle Scholar
  3. 3.
    Girosi, F., Regularization Theory??Radial Basis Functions and Networks,From Statistics to Neural Networks. Theory and Pattern Recognition Applications, J. H. Friedman, V. Cherkassky, and H. Wechsler (Eds.), Computer and Systems Sciences Series, Springer Verlag, Berlin, Germany, pp. 166–187, 1993.Google Scholar
  4. 4.
    Park, J., and Sandberg, I. W., Universal Approximation Using Radial-BasisFunction Networks, Neural Computation, Vol. 3, pp. 246–257, 1991.CrossRefGoogle Scholar
  5. 5.
    Gelfand, I. M., and Fomin, S. V., Calculus of Variations,Prentice Hall, Englewood Cliffs, New Jersey, 1963.Google Scholar
  6. 6.
    Sirisena, H. R., and Chou, F. S., Convergence of the Control Parametrization Ritz Method for Nonlinear Optimal Control Problems, Journal of Optimization Theory and Applications, Vol. 29, pp. 369–382, 1979.MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Kantorovich, L. V., and Krylov, V. I., Approximate Methods of Higher Analysis, P. Noordhoff Ltd., Groningen, The Netherlands, 1958.zbMATHGoogle Scholar
  8. 8.
    Beard, R. W., and Mclain, T. W., Successive Galerkin Approximation Algorithms for Nonlinear Optimal and Robust Control, International Journal of Control, Vol. 71, pp. 717–743, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Attouch, H., Variational Convergence for Functions and Operators, Pitman Publishing, London, England, 1984.Google Scholar
  10. 10.
    Sejnowski, T. J., and Rosenberg, C., Parallel Networks That Learn to Pronounce English Text, Complex Systems, Vol. 1, pp. 145–168, 1987.zbMATHGoogle Scholar
  11. 11.
    Zoppoli, R., and Parisini, T., Learning Techniques and Neural Networks for the Solution of N-Stage Nonlinear Nonquadratic Optimal Control Problems, Systems, Models and Feedback: Theory and Applications, Isidori and T. J. Tarn (Eds.), Birkhäuser, Boston, pp. 193–210, 1992.Google Scholar
  12. 12.
    Parisini, T., and Zoppoli, R., Neural Networks for Feedback Feedforward Nonlinear Control Systems,IEEE Transactions on Neural Networks, Vol. 5, pp. 436–449, 1994.CrossRefGoogle Scholar
  13. 13.
    Parisini, T., and Zoppoli, R., Neural Approximation for Multistage Optimal Control of Nonlinear Stochastic Systems, IEEE Transactions on Automatic Control, Vol. 41, pp. 889–895, 1996.MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Alessandri, A., Baglietto, M., Parisini, T., and Zoppoli, R., A Neural State Estimator with Bounded Errors for Nonlinear Systems,IEEE Transactions on Automatic Control, Vol. 44, pp. 2028–2042, 1999.MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Zoppoli, R., Sanguineti, M., and Parisini, T., Approximating Networks and the Extended Ritz Method for the Solution of Functional Optimization Problems, Journal of Optimization Theory and Applications, Vol. 112, pp. 403–439, 2002.MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Kolmogorov, A. N., Über die beste Annäherung von Funktionen Einer Gegebenen Funktionenklasse, Annals of Mathematics, Vol. 37, pp. 107–110, 1936 (English translation: On the Best Approximation of Functions of a Given Class, in Selected Works of A. N. Kolmogorov, Vol. I, V. M. Tikhomirov Ed., pp. 202–205. Kluwer, 1991).Google Scholar
  17. 17.
    Pinkus, A., n-Widths in Approximation Theory, Springer Verlag, New York, 1986.Google Scholar
  18. 18.
    Barron, A. R., Universal Approximation Bounds for Superpositions of a Sigmoidal Function, IEEE Transactions on Information Theory, Vol. 39, pp. 930–945, 1993.MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Kůrková, V., and Sanguineti, M., Comparison of Worst-Case Errors in Linear and Neural Network Approximation, IEEE Transactions on Information Theory, Vol. 48, pp. 264–275, 2002.zbMATHCrossRefGoogle Scholar
  20. 20.
    Kůrková, V., and Sanguineti, M., Bounds on Rates of Variable-Basis and Neural-Network Approximation, IEEE Transactions on Information Theory, Vol. 47, pp. 2659–2665, 2001.zbMATHCrossRefGoogle Scholar
  21. 21.
    Kůrková, V., Savický, P., and Hlavácková, K., Representations and Rates of Approximation of Real-Valued Boolean Functions by Neural Networks, Neural Networks, Vol. 11, pp. 651–659, 1998.CrossRefGoogle Scholar
  22. 22.
    Giulini, S., and Sanguineti, M., On Dimension-Independent Approximation by Neural Networks and Linear Approximators, Proceedings of the International Joint Conference on Neural Networks, Como, Italy, pp. I283–I288, 2000.Google Scholar
  23. 23.
    Girosi, F., Jones, M., and Poggio, T., Regularization Theory and Neural Networks Architectures, Neural Computation, Vol. 7, pp. 219–269, 1995.CrossRefGoogle Scholar
  24. 24.
    Jones, L.K., A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training, Annals of Statistics, Vol. 20, pp. 608–613, 1992.MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Breiman, L., Hinging Hyperplanes for Regression, Classification, and Function Approximation, IEEE Transactions on Information Theory, Vol. 39, pp. 993–1013, 1993.MathSciNetCrossRefGoogle Scholar
  26. 26.
    Mhaskar, H.N., Neural Networks for Optimal Approximation of Smooth and Analytic Functions, Neural Computation, Vol. 8, pp. 164–177, 1996.CrossRefGoogle Scholar
  27. 27.
    Girosi, F., and Anzellotti, G., Rates of Convergence of Approximation by Translates, A.I. Memo 1288, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 1992.Google Scholar
  28. 28.
    Kainen, P.C., Kůrková, V., and Sanguineti, M., Minimization of Error Functionals Under Weakened Compactness, submitted.Google Scholar
  29. 29.
    Kůrková, V., and Sanguineti, M., Error Estimates for Approximate Solution of Optimization Problems by Neural Networks, Workshop on Mathematical Diagnostics, Erice (Italy), 2002.Google Scholar
  30. 30.
    Kushner, H. J., and Yin, G. G., Stochastic Approximation Algorithms and Applications, Springer Verlag, New York, New York, 1997.zbMATHGoogle Scholar
  31. 31.
    Marschak, J., and Radner, R., Economic Theory of Teams, Yale University Press, 1972.zbMATHGoogle Scholar
  32. 32.
    Aicardi, M., Davoli, F., Minciardi, R., and Zoppoli, R., Decentralized Routing, Teams and Neural Networks in Communications, Proceedings 29th IEEE Conference on Decision and Control, pp. 2386–2390, 1990.Google Scholar
  33. 33.
    Parisini, T., and Zoppoli, R., Team Theory and Neural Networks for Dynamic Routing in Traffic and Communication Networks, Information and Decision Technologies, Vol. 19, pp. 1–18, 1993.zbMATHGoogle Scholar
  34. 34.
    Baglietto, M. Parisini, T., and Zoppoli, R., Distributed-Information Neural Control: The Case of Dynamic Routing in Traffic Networks, IEEE Transactions on Neural Networks, Vol. 12, pp. 485–502, 2001.CrossRefGoogle Scholar
  35. 35.
    Segall, A., The Modeling of Adaptive Routing in Data-Communication Networks, IEEE Transactions on Communications, Vol. 25, pp. 85–95, 1977.zbMATHCrossRefGoogle Scholar
  36. 36.
    Iftar, A., and Davison, E. J., A Decentralized Discrete-Time Controller for Dynamic Routing, International Journal of Control, Vol. 69, pp. 599–632, 1998.MathSciNetzbMATHCrossRefGoogle Scholar
  37. 37.
    Ho, Y. C., and Chu, K.C., Team Decision Theory and Information Structures in Optimal Control Problems - Part 1, IEEE Transactions on Automatic Control, Vol. 17, pp. 15–22, 1972.MathSciNetzbMATHCrossRefGoogle Scholar
  38. 38.
    Witsenhausen, H. S., A Counterexample in Stochastic Optimum Control, SIAM Journal on Control, Vol. 6, pp. 131–147, 1968.MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Baglietto, M., Parisini, T., and Zoppoli, R., Numerical Solutions to the Witsenhausen Counterexample by Approximating Networks, IEEE Transactions on Automatic Control, Vol. 46, pp. 1471–1477, 2001.MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Gallager, R. G., A Minimum Delay Routing Algorithm Using Distributed Computation, IEEE Transactions on Communications, Vol. 25, pp. 73–85, 1977.MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers B.V. 2003

Authors and Affiliations

  • M. Baglietto
    • 1
  • M. Sanguineti
    • 1
  • R. Zoppoli
    • 1
  1. 1.Department of CommunicationsComputer and System Sciences DIST - University of GenoaGenovaItaly

Personalised recommendations