Skip to main content
SpringerLink
Log in

Nonsmooth nonconvex optimization on Riemannian manifolds via bundle trust region algorithm

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

This paper develops an iterative algorithm to solve nonsmooth nonconvex optimization problems on complete Riemannian manifolds. The algorithm is based on the combination of the well known trust region and bundle methods. According to the process of the most bundle methods, the objective function is approximated by a piecewise linear working model which is updated by adding cutting planes at unsuccessful trial steps. Then at each iteration, by solving a subproblem that employs the working model in the objective function subject to the trust region, a candidate descent direction is obtained. We study the algorithm from both theoretical and practical points of view and its global convergence is verified to stationary points for locally Lipschitz functions. Moreover, in order to demonstrate the reliability and efficiency, a MATLAB implementation of the proposed algorithm is prepared and results of numerical experiments are reported.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availibility

Availability of supporting data not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Absil, P.A., Hosseini, S.: A collection of nonsmooth Riemannian optimization problems. In: Hosseini, S., Mordukhovich, B., Uschmajew, A. (eds.) Nonsmooth Optimization and Its Applications, International Series of Numerical Mathematics, vol. 170. Birkhuser, Cham (2019)

    Google Scholar 

  2. Absil, P.A., Baker, C., Gallivan, K.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7, 303–330 (2007)

    Article  MathSciNet  Google Scholar 

  3. Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008)

    Book  Google Scholar 

  4. Adler, R., Dedieu, J., Margulies, J., Martens, M., Shub, M.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22, 359–390 (2002)

    Article  MathSciNet  Google Scholar 

  5. Afsari, B., Tron, R., Vidal, R.: On the convergence of gradient descent for finding the Riemannian center of mass. SIAM J. Control Optim. 51, 2230–2260 (2013)

    Article  MathSciNet  Google Scholar 

  6. Apkarian, P., Noll, D., Ravanbod, L.: Nonsmooth bundle trust-region algorithm with applications to robust stability. Set-Valued Var. Anal. 24(1), 115–148 (2016)

    Article  MathSciNet  Google Scholar 

  7. Apkarian, P., Noll, D., Ravanbod, L.: Non-smooth optimization for robust control of infinite-dimensional systems. Set-Valued Var. Anal. 26(2), 405–429 (2018)

    Article  MathSciNet  Google Scholar 

  8. Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 29, 328–347 (2007)

    Article  MathSciNet  Google Scholar 

  9. Ashikhmin, A., Calderbank, A.: Grassmannian packings from operator Reed–Muller codes. IEEE Trans. Inf. Theory 56, 5689–5714 (2010)

    Article  MathSciNet  Google Scholar 

  10. Azagra, D., Ferrera, J., López-Mesas, F.: Nonsmooth analysis and Hamilton–Jacobi equations on Riemannian manifolds. J. Funct. Anal. 220, 304–361 (2005)

    Article  MathSciNet  Google Scholar 

  11. Baker, C., Absil, P., Gallivan, K.: An implicit trust-region method on Riemannian manifolds. IMA J. Numer. Anal. 28(4), 665–689 (2008)

    Article  MathSciNet  Google Scholar 

  12. Bento, G., Ferreira, O., Oliveira, P.: Local convergence of the proximal point method for a special class of nonconvex functions on Hadamard manifolds. Nonlinear Anal. Theory Methods Appl. 73, 564–572 (2010)

    Article  MathSciNet  Google Scholar 

  13. Bento, G., Ferreira, O., Oliveira, P.: Unconstrained steepest descent method for multicriteria optimization on Riemannian manifolds. J. Optim. Theory Appl. 154, 88–107 (2012)

    Article  MathSciNet  Google Scholar 

  14. Bento, G., Bitar, S., Neto, J.C., Oliveira, P., Souza, J.: Computing Riemannian center of mass on Hadamard manifolds. J. Optim. Theory Appl. 183, 977–992 (2019)

    Article  MathSciNet  Google Scholar 

  15. Bortoloti, M., Fernandes, T., Ferreira, O.: An efficient damped Newton-type algorithm with globalization strategy on Riemannian manifolds. J. Comput. Appl. Math. 403, 113853 (2022)

    Article  MathSciNet  Google Scholar 

  16. Boumal, N.: An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, Cambridge (2023)

    Book  Google Scholar 

  17. Clarke, F., Ledayaev, Y., Stern, R., Wolenki, P.: Nonsmooth Analysis and Control Theory. Springer, New York (1998)

    Google Scholar 

  18. Conn, A., Gould, N., Toint, P.: Trust Region Methods. Society for Industrial and Applied Mathematics, Philadelphia (2000)

    Book  Google Scholar 

  19. do Carmo, M.: Riemannian Geometry, Mathematics: Theory and Applications. Birkhäuser, Boston (1992)

    Book  Google Scholar 

  20. Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MathSciNet  Google Scholar 

  21. Dong, X., Frossard, P., Vandergheynst, P., Nefedov, N.: Clustering on multi-layer graphs via subspace analysis on Grassmann manifolds. IEEE Trans. Signal Process. 62, 905–918 (2014)

    Article  MathSciNet  Google Scholar 

  22. Ferreira, R., Xavier, J., Costeira, J., Barroso, V.: Newton method for Riemannian centroid computation in naturally reductive homogeneous spaces. In: IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, vol. 3, pp. III–III (2006)

  23. Ferreira, O., Silva, R.: Local convergence of Newton’s method under a majorant condition in Riemannian manifolds. IMA J. Numer. Anal. 32(4), 1696–1713 (2012)

    Article  MathSciNet  Google Scholar 

  24. Ferreira, O., Iusem, A., Németh, S.: Concepts and techniques of optimization on the sphere. TOP 22, 1148–1170 (2014)

    Article  MathSciNet  Google Scholar 

  25. Ferreira, O., Louzeiro, M., Prudente, L.: Gradient method for optimization on Riemannian manifolds with lower bounded curvature. SIAM J. Optim. 29(4), 2517–2541 (2019)

    Article  MathSciNet  Google Scholar 

  26. Fletcher, P., Venkatasubramanian, S., Joshi, S.: The geometric median on Riemannian manifolds with application to robust atlas estimation. NeuroImage 45(1), S143–S152 (2009)

    Article  Google Scholar 

  27. Grohs, P., Hosseini, S.: \(\varepsilon \)-subgradient algorithms for locally Lipschitz functions on Riemannian manifolds. Adv. Comput. Math. 42, 333–360 (2016)

    Article  MathSciNet  Google Scholar 

  28. Grohs, P., Hosseini, S.: Nonsmooth trust region algorithms for locally Lipschitz functions on Riemannian manifolds. IMA J. Numer. Anal. 36, 1167–1192 (2016)

    Article  MathSciNet  Google Scholar 

  29. Hare, W., Sagastizábal, C.: Computing proximal points of nonconvex functions. Math. Program. 116(1), 221–258 (2009)

    Article  MathSciNet  Google Scholar 

  30. Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)

    Article  MathSciNet  Google Scholar 

  31. Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonsmooth nonconvex functions with inexact information. Comput. Optim. Appl. 63(1), 1–28 (2016)

    Article  MathSciNet  Google Scholar 

  32. Hoseini, N., Nobakhtian, S.: A new trust region method for nonsmooth nonconvex optimization. Optimization 67, 1265–1286 (2018)

    Article  MathSciNet  Google Scholar 

  33. Hoseini-Monjezi, N.: A bundle trust region algorithm for minimizing locally Lipschitz functions. SIAM J. Optim. 33(1), 319–337 (2023)

    Article  MathSciNet  Google Scholar 

  34. Hoseini-Monjezi, N., Nobakhtian, S.: A new infeasible proximal bundle algorithm for nonsmooth nonconvex constrained optimization. Comput. Optim. Appl. 74(2), 443–480 (2019)

    Article  MathSciNet  Google Scholar 

  35. Hoseini-Monjezi, N., Nobakhtian, S.: Convergence of the proximal bundle algorithm for nonsmooth nonconvex optimization problems. Optim. Lett. 16, 1495–1511 (2022)

    Article  MathSciNet  Google Scholar 

  36. Hoseini-Monjezi, N., Nobakhtian, S.: An inexact multiple proximal bundle algorithm for nonsmooth nonconvex multiobjective optimization problems. Ann. Oper. Res. 311, 1123–1154 (2022)

    Article  MathSciNet  Google Scholar 

  37. Hoseini-Monjezi, N., Nobakhtian, S.: A proximal bundle-based algorithm for nonsmooth constrained multiobjective optimization problems with inexact data. Numer. Algorithms 89, 637–674 (2022)

    Article  MathSciNet  Google Scholar 

  38. Hoseini-Monjezi, N., Nobakhtian, S., Pouryayevali, M.R.: Proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds. IMA J. Numer. Anal. 43(1), 293–325 (2023)

    Article  MathSciNet  Google Scholar 

  39. Hosseini, S., Pouryayevali, M.: Generalized gradients and characterization of epi-Lipschitz sets in Riemannian manifolds. Nonlinear Anal. Theory Methods Appl. 74, 3884–3895 (2011)

    Article  MathSciNet  Google Scholar 

  40. Hosseini, S., Pouryayevali, M.: Euler characteristic of epi-Lipschitz subsets of Riemannian manifolds. J. Convex. Anal. 20(1), 67–91 (2013)

    MathSciNet  Google Scholar 

  41. Hosseini, S., Pouryayevali, M.: On the metric projection onto prox-regular subsets of Riemannian manifolds. Proc. Am. Math. Soc. 141, 233–244 (2013)

    Article  MathSciNet  Google Scholar 

  42. Hosseini, S., Uschmajew, A.: A Riemannian gradient sampling algorithm for nonsmooth optimization on manifolds. SIAM J. Optim. 27, 173–189 (2017)

    Article  MathSciNet  Google Scholar 

  43. Hosseini, S., Huang, W., Yousefpour, R.: Line search algorithms for locally Lipschitz functions on Riemannian manifolds. SIAM J. Optim. 28, 596–619 (2018)

    Article  MathSciNet  Google Scholar 

  44. Huang, W., Gallivan, K., Absil, P.A.: A Broyden class of quasi-Newton methods for Riemannian optimization. SIAM J. Optim. 25(3), 1660–1685 (2015)

    Article  MathSciNet  Google Scholar 

  45. Huang, W., Absil, P.A., Gallivan, K.: Intrinsic representation of tangent vector and vector transport on matrix manifolds. Numer. Math. 136, 523–543 (2017)

    Article  MathSciNet  Google Scholar 

  46. Kiwiel, K.: A linearization algorithm for nonsmooth minimization. Math. Oper. Res. 10(2), 185–194 (1985)

    Article  MathSciNet  Google Scholar 

  47. Lang, S.: Fundamentals of Differential Geometry. Graduate Texts in Mathematics. Springer, New York (1999)

    Book  Google Scholar 

  48. Lee, P.: Geometric Optimization for Computer Vision. Ph.D. Thesis, The Australian National University, Australia (2005)

  49. Lee, J.M.: Introduction to Riemannian Manifolds. Springer, Cham (2018)

    Book  Google Scholar 

  50. Lemaréchal, C.: An extension of Davidon methods to non-differentiable problems. Math. Program. Study 3, 95–109 (1975)

    Article  MathSciNet  Google Scholar 

  51. Li, C., Mordukhovich, B., Wang, J., Yao, J.: Weak sharp minima on Riemannian manifolds. SIAM J. Optim. 21(4), 1523–1560 (2011)

    Article  MathSciNet  Google Scholar 

  52. Li, X., Chen, S., Deng, Z., Qu, Q., Zhu, Z., So, A.M.C.: Weakly convex optimization over Stiefel manifold using Riemannian subgradient-type methods. SIAM J. Optim. 31(3), 1605–1634 (2021)

    Article  MathSciNet  Google Scholar 

  53. Noll, D.: Cutting plane oracles to minimize non-smooth non-convex functions. Set-Valued Var. Anal. 18(3–4), 531–568 (2010)

    Article  MathSciNet  Google Scholar 

  54. Noll, D., Prot, O., RondepierreA, A.: A proximity control algorithm to minimize nonsmooth and nonconvex functions. Pac. J. Optim. 4(3), 571–604 (2008)

    MathSciNet  Google Scholar 

  55. Ozbek, B., Ruyet, D.: Feedback Strategies for Wireless Communication. Springer, Berlin (2013)

    Google Scholar 

  56. Pennec, X., Fillard, P., Ayache, N.: A Riemannian framework for tensor computing. Int. J. Comput. Vis. 66, 41–66 (2006)

    Article  Google Scholar 

  57. Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program. 66, 25–43 (1994)

    Article  MathSciNet  Google Scholar 

  58. Qu, Q., Sun, J., Wright, J.: Finding a Sparse Vector in a Subspace: Linear Sparsity Using Alternating Directions, pp. 3401–3409. Red Hook, New York (2014)

    Google Scholar 

  59. Ring, W., Wirth, B.: Optimization methods on Riemannian manifolds and their application to shape space. SIAM J. Optim. 22, 596–627 (2012)

    Article  MathSciNet  Google Scholar 

  60. Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2, 121–152 (1992)

    Article  MathSciNet  Google Scholar 

  61. Weinmann, A., Demaret, L., Storath, M.: Total variation regularization for manifold-valued data. SIAM J. Imaging Sci. 7, 2226–2257 (2014)

    Article  MathSciNet  Google Scholar 

  62. Wolfe, P.: An extension of Davidon methods to non-differentiable problems. Math. Program. Study 3, 145–173 (1975)

    Article  MathSciNet  Google Scholar 

  63. Yang, Y., Pang, L., Ma, X., Shen, J.: Constrained nonconvex nonsmooth optimization via proximal bundle method. J. Optim. Theory Appl. 163(3), 900–925 (2014)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors appreciate the careful reading and helpful comments and suggestions of the coordinating editor and two anonymous referees.

Funding

This research was not supported by any grant.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics and Computer Science, Faculty of Mathematics and Statistics, University of Isfahan, Isfahan, Iran

    N. Hoseini Monjezi & S. Nobakhtian

  2. Department of Pure Mathematics, Faculty of Mathematics and Statistics, University of Isfahan, Isfahan, Iran

    M. R. Pouryayevali

Authors
  1. N. Hoseini Monjezi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Nobakhtian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. R. Pouryayevali
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The paper is written in a group of three, and all parts of the research work (design, analysis and implementation of the algorithm, writing and editing the first draft of the manuscript, coding and methodology) were done by all three together. Three authors read and approved the final manuscript.

Corresponding author

Correspondence to M. R. Pouryayevali.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Human and animal rights

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoseini Monjezi, N., Nobakhtian, S. & Pouryayevali, M.R. Nonsmooth nonconvex optimization on Riemannian manifolds via bundle trust region algorithm. Comput Optim Appl (2024). https://doi.org/10.1007/s10589-024-00569-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10589-024-00569-5

Keywords

Mathematics Subject Classification

Search

Navigation