Skip to main content
SpringerLink
Log in

Accelerated derivative-free nonlinear least-squares applied to the estimation of Manning coefficients

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

A general framework for solving nonlinear least squares problems without the employment of derivatives is proposed in the present paper together with a new general global convergence theory. With the aim to cope with the case in which the number of variables is big (for the standards of derivative-free optimization), two dimension-reduction procedures are introduced. One of them is based on iterative subspace minimization and the other one is based on spline interpolation with variable nodes. Each iteration based on those procedures is followed by an acceleration step inspired in the Sequential Secant Method. The practical motivation for this work is the estimation of parameters in Hydraulic models applied to dam breaking problems. Numerical examples of the application of the new method to those problems are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability statement

The authors confirm that all data generated or analyzed in the development of this work are adequately included or referenced in the article itself. In particular, source codes are freely available for download at https://www.ime.usp.br/~egbirgin/.

Notes

  1. Provided by Hongchao Zhang on January 11th, 2021.

References

  1. Anderson, D.G.: Iterative procedures for nonlinear integral equations. J. Assoc. Comput. Mach. 12, 547–560 (1965)

    Article  MathSciNet  Google Scholar 

  2. Audet, Ch., Dennis, J.E., Jr.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188–217 (2006). https://doi.org/10.1137/040603371

    Article  MathSciNet  MATH  Google Scholar 

  3. Barnes, J.G.P.: An algorithm for solving nonlinear equations based on the secant method. Comput. J. 8, 66–72 (1965)

    Article  MathSciNet  Google Scholar 

  4. Birgin, E. G., Martínez, J. M.: Secant acceleration of sequential residual methods for large scale nonlinear systems of equations (2020) arXiv:2012.13251v1

  5. Boutet, N., Haelterman, R., Degroote, J.: Secant update version of quasi-Newton PSB with weighted multisecant equations. Comput Optim Appl 75(2), 1–26 (2020)

    Article  MathSciNet  Google Scholar 

  6. Boutet, N., Haelterman, R., Degroote, J.: Secant update generalized version of PSB: a new approach. Comput Optim Appl 78, 953–982 (2021)

    Article  MathSciNet  Google Scholar 

  7. Brezinski, C.: Convergence acceleration during the 20th century. J Comput Appl Math 122, 1–21 (2000)

    Article  MathSciNet  Google Scholar 

  8. Brezinski, C., Redivo-Zaglia, M.: Extrapolation methods theory and practice. North-Holland, Amsterdam (1991)

    MATH  Google Scholar 

  9. Brezinski, C., Redivo-Zaglia, M., Saad, Y.: Shanks sequence transformations and Anderson acceleration. SIAM Rev 60, 646–669 (2018)

    Article  MathSciNet  Google Scholar 

  10. Cartis, C., Roberts, L.: A derivative-free Gauss-Newton method. Math Program Comput 11, 631–674 (2019)

    Article  MathSciNet  Google Scholar 

  11. Cartis, C., Roberts, L.: Scalable subspace methods for derivative-free nonlinear least-squares optimization (2021) arXiv:2102.12016

  12. Chorobura, F.: Worst-case complexity analysis of derivative-free nonmonotone methods for solving nonlinear systems of equations. Master Dissertation, Federal University of Paraná, Curitiba, PR, Brazil, (2020)

  13. Conn, A. R., Scheinberg, K., Vicente, L. N.: Introduction to Derivative-Free Optimization, MPS-SIAM Series on Optimization, (2009)

  14. Ding, Y., Jia, Y., Wang, S.S.Y.: Identification of Manning’s roughness coefficients in shallow water flows. J. Hydr. Eng. 130(6), 501–510 (2004)

    Article  Google Scholar 

  15. Graf, W.H., Altinakar, M.S.: Hydraulique Fluviale - Tome 1: Ecoulement permanent uniforme et non uniforme. Presses Polytechniques e Universitaires Romandes, Lausanne (1993)

    Google Scholar 

  16. Fang, H.R., Saad, Y.: Two classes of multisecant methods for nonlinear acceleration. Numer. Lin. Algebra Appl. 16, 197–221 (2009)

    Article  MathSciNet  Google Scholar 

  17. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Res. Logistics Q. 3, 95–110 (1956)

    Article  MathSciNet  Google Scholar 

  18. Gratton, S., Toint, Ph.L.: Multi-secant equations, approximate invariant subspaces and multigrid optimization, Technical Report 07/11. Department of Mathematics. University of Namur - FUNDP, Namur, Belgium (2007)

    Google Scholar 

  19. Gratton, S., Malmedy, V., Toint, Ph.L.: Quasi-Newton updates with weighted secant equations. Optim. Methods Soft. 30, 748–755 (2015)

    Article  MathSciNet  Google Scholar 

  20. Haelterman, R., Bogaers, A., Degroote, J., Boutet, N.: Quasi-Newton methods for the acceleration of multi-physics codes. IAENG Int. J. Appl. Math. 47, 352–360 (2017)

    MathSciNet  Google Scholar 

  21. Ho, N., Olson, S.D., Walker, H.F.: Accelerating the Uzawa algorithm. SIAM J. Sci. Comput. 39, 461–476 (2017)

    Article  MathSciNet  Google Scholar 

  22. Jankowska, J.: Theory of multivariate secant methods. SIAM J. Num. Anal. 16, 547–562 (1979)

    Article  MathSciNet  Google Scholar 

  23. La Cruz, W., Martínez, J.M., Raydan, M.: Spectral residual method without gradient information for solving large-scale nonlinear systems of equations. Math. Comput. 75, 1429–1448 (2006)

    Article  MathSciNet  Google Scholar 

  24. La Cruz, W., Raydan, M.: Nonmonotone Spectral Methods for Large-Scale Nonlinear Systems. Optim. Methods Soft. 18, 583–599 (2003)

    Article  MathSciNet  Google Scholar 

  25. LeVeque, R.J.: Numerical Methods for Conservation Laws. Lectures in Mathematics, ETH Zürich, Birkäuser (1992)

  26. Martini dos Santos, T., Reips, L., Martínez, J.M.: Under-relaxed quasi-Newton acceleration for an inverse fixed-point problem coming from positron-emission tomography. J. Inverse Ill-Posed Prob. 26, 755–770 (2018)

    Article  MathSciNet  Google Scholar 

  27. Meli, E., Morini, B., Porcelli, M., Sgattoni, C.: Solving nonlinear systems of equations via spectral residual methods: stepsize selection and applications (2020) arXiv:2005.05851v2

  28. Ni, P., Walker, H.F.: Anderson acceleration for fixed-point iterations. SIAM J. Num. Anal. 49, 1715–1735 (2011)

    Article  MathSciNet  Google Scholar 

  29. Ortega, J.. M., Rheinboldt, W.. C.: Iterative solution of nonlinear equations in several variables. Academic Press, Cambridge (1970)

    MATH  Google Scholar 

  30. Porto, R.M.: Hidráulica Básica. EESC-USP, São Carlos, SP, Brazil (2004)

    Google Scholar 

  31. Powell, M.J.D.: UOBYQA, Unconstrained optimization by quadratic approximation. Math. Program. 92, 555–582 (2002)

    Article  MathSciNet  Google Scholar 

  32. Powell, M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. 100, 183–215 (2004)

    Article  MathSciNet  Google Scholar 

  33. Powell, M.J.D.: Beyond symmetric Broyden for updating quadratic models in minimization without derivatives. Math. Program. 138, 475–500 (2013)

    Article  MathSciNet  Google Scholar 

  34. Powell, M. J. D.: The BOBYQA algorithm for bound constrained optimization without derivatives, Report No. DAMTP 2009/NA06, Centre for Mathematical Sciences, University of Cambridge, (2009)

  35. Ralston, M.L., Jennrich, R.I.: Dud, a derivative free algorithm for nonlinear least squares. Technometrics 20, 7–14 (1978)

    Article  Google Scholar 

  36. Rohwedder, T., Schneider, R.: An analysis for the DIIS acceleration method used in quantum chemistry calculations. J. Math. Chem. 49, 1889 (2011)

    Article  MathSciNet  Google Scholar 

  37. Saint-Venant, A.J.C.: Théorie du mouvement non-permanent des eaux, avec application aux crues des rivière at à l’introduction des marées dans leur lit. Comptes Rendus des Séances de Académie des Sci. 73, 147–154 (1871)

    MATH  Google Scholar 

  38. Scheufele, K., Mell, M.: Robust multisecant Quasi-Newton variants for parallel fluid-structure simulations-and other multiphysics applications. SIAM J. Sci. Comput. 39, 404–433 (2017)

    Article  MathSciNet  Google Scholar 

  39. Schnabel, R.B.: Quasi-Newton methods using multiple secant equations, Technical Report CU-CS-247-83. Deptartment of Computer Science. University of Colorado, Boulder, CO, USA (1983)

    Book  Google Scholar 

  40. Varadhan, R., Gilbert, P.D.: BB An R package for solving a large system of nonlinear equations and for optimizing a high-dimensional nonlinear objective function. J. Stat. Soft. 32, 4 (2009)

    Article  Google Scholar 

  41. Walker, H. F., Woodward, C. S., Yang, U. M.: An accelerated fixed-point iteration for solution of variably saturated flow, in Proceedings of the XVIII International Conference on Water Resources, CMWR 2010, J. Carrera, ed., CIMNE, Barcelona, (2010) (available online at http://congress.cimne.com/CMWR2010/Proceedings/Start.html)

  42. Wang, Z., Wen, Z., Yuan, Y.-X.: A subspace trust region method for large scale unconstrained optimization, in Numerical Linear Algebra and Optimization, Ya-Xiang Yuan ed., Science Press, (2004), pp. 264–274

  43. Wild, S.M.: Solving derivative-free nonlinear least squares problems with POUNDERS. In: Terlaky, T., Anjos, M.F., Ahmed, S. (eds.) Advances and trends in optimization with engineering applications, pp. 529–540. SIAM, Philadephia, PA, USA (2017)

    Chapter  Google Scholar 

  44. Wolfe, P.: The secant method for simultaneous nonlinear equations. Commun. ACM 2, 12–13 (1959)

    Article  Google Scholar 

  45. Yuan, Y.-X.: Subspace methods for large scale nonlinear equations and nonlinear least squares. Optim. Eng. 10, 207–218 (2009)

    Article  MathSciNet  Google Scholar 

  46. Zeev, N., Savasta, O., Cores, D.: Nonmonotone spectral projected gradient method applied to full waveform inversion. Geophys. Prospect. 54, 525–534 (2006)

    Article  Google Scholar 

  47. Zhang, H., Conn, A.R.: On the local convergence of a derivative-free algorithm for least-squares minimization. Comput. Optim. Appl. 51, 481–507 (2012)

    Article  MathSciNet  Google Scholar 

  48. Zhang, H., Conn, A.R., Scheinberg, K.: A derivative-free algorithm for least-squares minimization. SIAM J. Optim. 20, 3555–3576 (2010)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Institute of Mathematics and Statistics, University of São Paulo, Rua do Matão, 1010, Cidade Universitária, 05508-090, São Paulo, SP, Brazil

    E. G. Birgin

  2. Department of Applied Mathematics, Institute of Mathematics, Statistics, and Scientific Computing (IMECC), State University of Campinas, 13083-859, Campinas, SP, Brazil

    J. M. Martínez

Authors
  1. E. G. Birgin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. M. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to E. G. Birgin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by FAPESP (grants 2013/07375-0, 2016/01860-1, and 2018/24293-0) and CNPq (grants 302538/2019-4 and 302682/2019-8).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Birgin, E.G., Martínez, J.M. Accelerated derivative-free nonlinear least-squares applied to the estimation of Manning coefficients. Comput Optim Appl 81, 689–715 (2022). https://doi.org/10.1007/s10589-021-00344-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-021-00344-w

Keywords

Search

Navigation