Skip to main content
SpringerLink
Log in

From Functional Optimization to Nonlinear Programming by the Extended Ritz Method

  • Chapter
  • First Online:
Neural Approximations for Optimal Control and Decision

Part of the book series: Communications and Control Engineering ((CCE))

  • 639 Accesses

Abstract

This chapter describes the approximate solution of infinite-dimensional optimization problems by the “Extended Ritz Method” (ERIM). The ERIM consists in substituting the admissible functions with fixed-structure parametrized (FSP) functions containing vectors of “free” parameters. The larger the dimensions, the more accurate the approximations of the optimal solutions of the original functional optimization problems. This requires solving easier nonlinear programming problems. In the area of function approximation, we review the definition of approximating sequences of sets, which enjoy the property of density in the sets of functions one wants to approximate. Then, we provide the definition of polynomially complex approximating sequences of sets, which are able to approximate functions provided with suitable regularity properties by using, for a desired arbitrary accuracy, a number of “free” parameters increasing at most polynomially when the number of function arguments grows. In the less studied area of approximate solution of infinite-dimensional optimization problems, the optimizing sequences and the polynomially complex optimizing sequences of FSP functions are defined. Results are presented that allow to conclude that, if appropriate hypotheses occur, polynomially complex approximating sequences of sets give rise to polynomially complex optimizing sequences of FSP functions, possibly mitigating the curse of dimensionality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Four sequences of approximating FSP functions are defined and discussed. Such sequences are connected to each other by properties that the reader may find somewhat difficult and tedious. To better understand the relationships among the four families, the reader may refer to Tables 2.1 and 2.2, and to Fig. 2.10.

  2. 2.

    The reader should now understand why Remark on Notation 2.1 is useful to simplify the notation; see Point 3 in the remark.

  3. 3.

    Note that one should specify the space \({\mathscr {G}}^d\) in the definition, since the same set \({\mathcal {M}}^d\) might be considered as subset of various normed linear spaces. We refer the reader to Chap. 3 for a more detailed treatment.

  4. 4.

    In Remark 2.2, we stressed the meaning of the term “absolute constant,” i.e., a constant that does not depend on any other quantity involved in the context at issue. As regards \({\tau }\), \({\kappa }_1^-, {\kappa }_1^+,\kappa _2^-, {\kappa }_2^+\), we have to point out that such “constants” are not absolute. Indeed, they may depend on d.

  5. 5.

    The functions \(L_{{\tau }}\) and \(U_{{\tau }}\) take on the form of the “comparison functions” of class \({{\mathcal {K}}}_{\infty }\) typically used in Lyapunov stability analysis of nonlinear dynamic systems (see, e.g., [30]).

  6. 6.

    It is worth mentioning also the work [31], where suboptimal feedback control laws are searched for dynamic systems under LQ assumptions via the Ritz method. However, in [31] the point of view is different: the authors aim at deriving, for an optimal control problem whose solution is known in closed form, an approximate control law taking on a simple structure.

References

  1. Alessandri A, Cervellera C, Sanguineti M (2007) Functional optimal estimation problems and their solution by nonlinear approximation schemes. J Optim Theory Appl 134:445–466

    Article  MathSciNet  Google Scholar 

  2. Alessandri A, Gnecco G, Sanguineti M (2010) Minimizing sequences for a family of functional optimal estimation problems. J Optim Theory Appl 147:243–262

    Article  MathSciNet  Google Scholar 

  3. Alessandri A, Gaggero M, Zoppoli R (2012) Feedback optimal control of distributed parameter systems by using finite-dimensional approximation schemes. IEEE Trans Neural Netw Learn Syst 23:984–996

    Article  Google Scholar 

  4. Alt W (1984) On the approximation of infinite optimization problems with an application to optimal control problems. Appl Math Optim 12:15–27

    Article  MathSciNet  Google Scholar 

  5. Barron AR (1992) Neural net approximation. In: Narendra KS (ed) Proceedings of the 7th Yale workshop on adaptive and learning systems. Yale University Press, pp 69–72

    Google Scholar 

  6. Barron AR (1993) Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans Inf Theory 39:930–945

    MathSciNet  MATH  Google Scholar 

  7. Breiman L (1993) Hinging hyperplanes for regression, classification, and function approximation. IEEE Trans Inf Theory 39:993–1013

    Article  MathSciNet  Google Scholar 

  8. Dal Maso G (1993) Introduction to gamma-convergence. Springer

    Google Scholar 

  9. Darken C, Donahue M, Gurvits L, Sontag E (1993) Rate of approximation results motivated by robust neural network learning. In: Proceedings of the sixth annual ACM conference on computational learning theory. ACM, pp 303–309

    Google Scholar 

  10. Donahue M, Gurvits L, Darken C, Sontag E (1997) Rates of convex approximation in non-Hilbert spaces. Constr Approx 13:187–220

    Article  MathSciNet  Google Scholar 

  11. Gaggero M, Gnecco G, Sanguineti M (2014) Suboptimal policies for stochastic \(N\)-stage optimization problems: accuracy analysis and a case study from optimal consumption. In: El Ouardighi F, Kogan K (eds) Models and methods in economics and management. Springer, pp 27–50

    Google Scholar 

  12. Gelfand IM, Fomin SV (1963) Calculus of variations. Prentice Hall

    Google Scholar 

  13. Girosi F, Poggio T (1990) Networks and the best approximation property. Biol Cybern 63:169–176

    Article  MathSciNet  Google Scholar 

  14. Girosi F, Anzellotti G (1993) Rates of convergence for Radial Basis Functions and neural networks. In: Mammone RJ (ed) Artificial neural networks for speech and vision. Chapman & Hall, pp 97–113

    Google Scholar 

  15. Giulini S, Sanguineti M (2009) Approximation schemes for functional optimization problems. J Optim Theory Appl 140:33–54

    Article  MathSciNet  Google Scholar 

  16. Gnecco G, Sanguineti M (2009) Accuracy of suboptimal solutions to kernel principal component analysis. Comput Optim Appl 42:265–287

    Article  MathSciNet  Google Scholar 

  17. Gnecco G, Sanguineti M (2010) Error bounds for suboptimal solutions to kernel principal component analysis. Optim Lett 4:197–210

    Article  MathSciNet  Google Scholar 

  18. Gnecco G, Sanguineti M (2010) Estimates of variation with respect to a set and applications to optimization problems. J Optim Theory Appl 145:53–75

    Article  MathSciNet  Google Scholar 

  19. Gnecco G, Sanguineti M (2010) Regularization techniques and suboptimal solutions to optimization problems in learning from data. Neural Comput 22:793–829

    Article  MathSciNet  Google Scholar 

  20. Gnecco G, Sanguineti M (2010) Suboptimal solutions to dynamic optimization problems via approximations of the policy functions. J Optim Theory Appl 146:764–794

    Article  MathSciNet  Google Scholar 

  21. Gnecco G, Sanguineti M (2011) Team optimization problems with Lipschitz continuous strategies. Optim Lett 5:333–346

    Article  MathSciNet  Google Scholar 

  22. Gnecco G, Sanguineti M (2012) New insights into Witsenhausen’s counterexample. Optim Lett 6:1425–1446

    Article  MathSciNet  Google Scholar 

  23. Gnecco G, Sanguineti M, Gaggero M (2012) Suboptimal solutions to team optimization problems with stochastic information structure. SIAM J Optim 22:212–243

    Article  MathSciNet  Google Scholar 

  24. Gurvits L, Koiran P (1997) Approximation and learning of convex superpositions. J Comput Syst Sci 55:161–170

    Article  MathSciNet  Google Scholar 

  25. Jones LK (1992) A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann Stat 20:608–613

    Article  MathSciNet  Google Scholar 

  26. Kainen P, Kůrková V, Sanguineti M (2003) Minimization of error functionals over variable-basis functions. SIAM J Optim 14:732–742

    Article  MathSciNet  Google Scholar 

  27. Kainen PC, Kůrková V (2005) Rates of minimization of error functionals over Boolean variable-basis functions. J Math Model Algorithms 4:355–368

    Article  MathSciNet  Google Scholar 

  28. Kainen PC, Kůrková V (2009) Complexity of Gaussian radial basis networks approximating smooth functions. J Complex 25:63–74

    Article  MathSciNet  Google Scholar 

  29. Kainen PC, Kůrková V (2012) Dependence of computational models on input dimension: tractability of approximation and optimization tasks. IEEE Trans Inf Theory 58:1203–1214

    Article  MathSciNet  Google Scholar 

  30. Khalil HK (1996) Nonlinear systems. Prentice-Hall

    Google Scholar 

  31. Kleinman DL, Athans M (1968) The design of suboptimal linear time-varying systems. IEEE Trans Autom Control 13:150–159

    Article  MathSciNet  Google Scholar 

  32. Knuth DE (1976) Big omicron and big omega and big theta. SIGACT News

    Google Scholar 

  33. Kůrková V (2008) Minimization of error functionals over perceptron networks. Neural Comput 20:252–270

    Article  MathSciNet  Google Scholar 

  34. Kůrková V, Sanguineti M (2005) Error estimates for approximate optimization by the Extended Ritz Method. SIAM J Optim 15:461–487

    Article  MathSciNet  Google Scholar 

  35. Kůrková V, Savický P, Hlavácková K (1998) Representations and rates of approximation of real-valued Boolean functions by neural networks. Neural Netw 11:651–659

    Article  Google Scholar 

  36. Makovoz Y (1996) Random approximants and neural networks. J Approx Theory 85:98–109

    Article  MathSciNet  Google Scholar 

  37. Mhaskar HN (2004) On the tractability of multivariate integration and approximation by neural networks. J Complex 20:561–590

    Article  MathSciNet  Google Scholar 

  38. Parisini T, Zoppoli R (2009) Connections between approximating sequences and optimizing sequences in the Extended Ritz Method. Technical Report 2009.1, DIPTEM, Università di Genova

    Google Scholar 

  39. Traub JF, Werschulz AG (1999) Complexity and information. Cambridge University Press

    Google Scholar 

  40. Wasilkowski GW, Woźniakowski H (2001) Complexity of weighted approximation over \(\Re ^d\). J Complex 17:722–740

    Article  MathSciNet  Google Scholar 

  41. Woźniakowski H (1994) Tractability and strong tractability of linear multivariate problems. J Complex 10:96–128

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DIBRIS, Università di Genova, Genoa, Italy

    Riccardo Zoppoli & Marcello Sanguineti

  2. AXES Research Unit, IMT—School of Advanced Studies Lucca, Lucca, Italy

    Giorgio Gnecco

  3. Imperial College London, London, UK

    Thomas Parisini

  4. University of Trieste, Trieste, Italy

    Thomas Parisini

Authors
  1. Riccardo Zoppoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcello Sanguineti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giorgio Gnecco
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Parisini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riccardo Zoppoli .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zoppoli, R., Sanguineti, M., Gnecco, G., Parisini, T. (2020). From Functional Optimization to Nonlinear Programming by the Extended Ritz Method. In: Neural Approximations for Optimal Control and Decision. Communications and Control Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-29693-3_2

Download citation

Publish with us

Policies and ethics

Search

Navigation