Abstract
This chapter considers optimization problems on Riemannian manifolds and presents asymptotic and iteration-complexity analysis for gradient and subgradient methods on manifolds with sectional curvatures bounded from below. It also establishes asymptotic and iteration-complexity analysis for the proximal point method on Hadamard manifolds.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Absil, P.A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)
Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2008). With a foreword by Paul Van Dooren
Afsari, B., Tron, R., Vidal, R.: On the convergence of gradient descent for finding the Riemannian center of mass. SIAM J. Control Optim. 51(3), 2230–2260 (2013)
Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J. Matrix Anal. Appl. 29(1), 328–347 (2006)
Bačák, M.: The proximal point algorithm in metric spaces. Israel J. Math. 194(2), 689–701 (2013)
Bačák, M., Bergmann, R., Steidl, G., Weinmann, A.: A second order nonsmooth variational model for restoring manifold-valued images. SIAM J. Sci. Comput. 38(1), A567–A597 (2016)
Batista, E.E.A., Bento, G.d.C., Ferreira, O.P.: Enlargement of monotone vector fields and an inexact proximal point method for variational inequalities in Hadamard manifolds. J. Optim. Theory Appl. 170(3), 916–931 (2016)
Baust, M., Weinmann, A., Wieczorek, M., Lasser, T., Storath, M., Navab, N.: Combined tensor fitting and TV regularization in diffusion tensor imaging based on a Riemannian manifold approach. IEEE Trans. Med. Imaging 35(8), 1972–1989 (2016)
Bento, G.C., Cruz Neto, J.X.: A subgradient method for multiobjective optimization on Riemannian manifolds. J. Optim. Theory Appl. 159(1), 125–137 (2013)
Bento, G.C., Cruz Neto, J.X.: Finite termination of the proximal point method for convex functions on Hadamard manifolds. Optimization 63(9), 1281–1288 (2014)
Bento, G.C., Melo, J.G.: Subgradient method for convex feasibility on Riemannian manifolds. J. Optim. Theory Appl. 152(3), 773–785 (2012)
Bento, G.C., Ferreira, O.P., Oliveira, P.R.: Proximal point method for a special class of nonconvex functions on Hadamard manifolds. Optimization 64(2), 289–319 (2015)
Bento, G., da Cruz Neto, J., Oliveira, P.R.: A new approach to the proximal point method: convergence on general Riemannian manifolds. J. Optim. Theory Appl. 168(3), 743–755 (2016)
Bento, G.C., Ferreira, O.P., Melo, J.G.: Iteration-complexity of gradient, subgradient and proximal point methods on Riemannian manifolds. J. Optim. Theory Appl. 173(2), 548–562 (2017)
Bento, G.C., Bitar, S.D.B., Cruz Neto, J.X., Oliveira, P.R., Souza, J.C.: Computing Riemannian center of mass on Hadamard manifolds. J. Optim. Theory Appl. (2019)
Bergmann, R., Weinmann, A.: A second-order TV-type approach for inpainting and denoising higher dimensional combined cyclic and vector space data. J. Math. Imaging Vision 55(3), 401–427 (2016)
Bergmann, R., Persch, J., Steidl, G.: A parallel Douglas-Rachford algorithm for minimizing ROF-like functionals on images with values in symmetric Hadamard manifolds. SIAM J. Imaging Sci. 9(3), 901–937 (2016)
Bhattacharya, A., Bhattacharya, R.: Statistics on Riemannian manifolds: asymptotic distribution and curvature. Proc. Amer. Math. Soc. 136(8), 2959–2967 (2008)
Bini, D.A., Iannazzo, B.: Computing the Karcher mean of symmetric positive definite matrices. Linear Algebra Appl. 438(4), 1700–1710 (2013)
Bonnabel, S.: Stochastic gradient descent on Riemannian manifolds. IEEE Trans. Automat. Control 58(9), 2217–2229 (2013)
Boumal, N., Absil, P.A., Cartis, C.: Global rates of convergence for nonconvex optimization on manifolds. IMA J. Numer. Anal. 39(1), 1–33 (2018)
Bredies, K., Holler, M., Storath, M., Weinmann, A.: Total generalized variation for manifold-valued data. SIAM J. Imaging Sci. 11(3), 1785–1848 (2018)
Burachik, R., Drummond, L.M.G., Iusem, A.N., Svaiter, B.F.: Full convergence of the steepest descent method with inexact line searches. Optimization 32(2), 137–146 (1995)
Chaipunya, P., Kumam, P.: On the proximal point method in Hadamard spaces. Optimization 66(10), 1647–1665 (2017)
Cholamjiak, P., Abdou, A.A.N., Cho, Y.J.: Proximal point algorithms involving fixed points of nonexpansive mappings in CAT(0) spaces. Fixed Point Theory Appl. 2015(13), 227 (2015)
Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2, Ser. B), 261–275 (1993)
Cuntavepanit, A., Phuengrattana, W.: On solving the minimization problem and the fixed-point problem for a finite family of non-expansive mappings in CAT(0) spaces. Optim. Methods Softw. 33(2), 311–321 (2018)
da Cruz Neto, J.X., de Lima, L.L., Oliveira, P.R.: Geodesic algorithms in Riemannian geometry. Balkan J. Geom. Appl. 3(2), 89–100 (1998)
da Cruz Neto, J.X., Ferreira, O.P., Lucambio Pérez, L.R.: Contributions to the study of monotone vector fields. Acta Math. Hungar. 94(4), 307–320 (2002)
Da Cruz Neto, J.X., Ferreira, O.P., Pérez, L.R.L., Németh, S.Z.: Convex- and monotone-transformable mathematical programming problems and a proximal-like point method. J. Global Optim. 35(1), 53–69 (2006)
do Carmo, M.P.: Riemannian Geometry. Mathematics: Theory & Applications. Birkhäuser Boston, Boston (1992). Translated from the second Portuguese edition by Francis Flaherty
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1999)
Esposito, M., Hennersperger, C., Gobl, R., Demaret, L., Storath, M., Navab, N., Baust, M., Weinmann, A.: Total variation regularization of pose signals with an application to 3D freehand ultrasound. IEEE Trans. Med. Imaging 38(10), 2245–2258 (2019)
Ferreira, O.P., Oliveira, P.R.: Subgradient algorithm on Riemannian manifolds. J. Optim. Theory Appl. 97(1), 93–104 (1998)
Ferreira, O.P., Oliveira, P.R.: Proximal point algorithm on Riemannian manifolds. Optimization 51(2), 257–270 (2002)
Ferreira, O.P., Louzeiro, M.S., Prudente, L.F.: Gradient method for optimization on riemannian manifolds with lower bounded curvature. SIAM J. Optim. 29(4), 2517–2541 (2019). e-prints. arXiv:1806.02694
Ferreira, O.P., Louzeiro, M.S., Prudente, L.F.: Iteration-complexity of the subgradient method on Riemannian manifolds with lower bounded curvature. Optimization 68(4), 713–729 (2019)
Fletcher, P.T.: Geodesic regression and the theory of least squares on Riemannian manifolds. Int. J. Comput. Vis. 105(2), 171–185 (2013)
Freifeld, O., Black, M.J.: Lie bodies: a manifold representation of 3D human shape. In: Proceedings of European Conference on Computer Vision 2012. Springer, Berlin (2012)
Gabay, D.: Minimizing a differentiable function over a differential manifold. J. Optim. Theory Appl. 37(2), 177–219 (1982)
Goffin, J.L.: Subgradient optimization in nonsmooth optimization (including the Soviet revolution). Doc. Math. (Extra vol.: Optimization stories), 277–290 (2012)
Grohs, P., Hosseini, S.: ε-Subgradient algorithms for locally lipschitz functions on Riemannian manifolds. Adv. Comput. Math. 42(2), 333–360 (2016)
Hawe, S., Kleinsteuber, M., Diepold, K.: Analysis operator learning and its application to image reconstruction. IEEE Trans. Image Process. 22(6), 2138–2150 (2013)
Honkela, A., Raiko, T., Kuusela, M., Tornio, M., Karhunen, J.: Approximate Riemannian conjugate gradient learning for fixed-form variational Bayes. J. Mach. Learn. Res. 11, 3235–3268 (2010)
Huang, W., Gallivan, K.A., Absil, P.A.: A Broyden class of quasi-Newton methods for Riemannian optimization. SIAM J. Optim. 25(3), 1660–1685 (2015)
Jeuris, B., Vandebril, R., Vandereycken, B.: A survey and comparison of contemporary algorithms for computing the matrix geometric mean. Electron. Trans. Numer. Anal. 39, 379–402 (2012)
Kajimura, T., Kimura, Y.: Resolvents of convex functions in complete geodesic metric spaces with negative curvature. J. Fixed Point Theory Appl. 21(1), 15 (2019). Art. 32
Karmarkar, N.: Riemannian geometry underlying interior-point methods for linear programming. In: Mathematical Developments Arising from Linear Programming (Brunswick, ME, 1988), Contemporary Mathematics, vol. 114, pp. 51–75. American Mathematical Society, Providence (1990)
Kum, S., Yun, S.: Incremental gradient method for Karcher mean on symmetric cones. J. Optim. Theory Appl. 172(1), 141–155 (2017)
Lang, S.: Fundamentals of Differential Geometry, Graduate Texts in Mathematics, vol. 191. Springer, New York (1999)
Lerkchaiyaphum, K., Phuengrattana, W.: Iterative approaches to solving convex minimization problems and fixed point problems in complete CAT(0) spaces. Numer. Algorithms 77(3), 727–740 (2018)
Leuştean, L., Nicolae, A., Sipoş, A.: An abstract proximal point algorithm. J. Global Optim. 72(3), 553–577 (2018)
Li, C., Yao, J.C.: Variational inequalities for set-valued vector fields on Riemannian manifolds: convexity of the solution set and the proximal point algorithm. SIAM J. Control Optim. 50(4), 2486–2514 (2012)
Li, C., López, G., Martín-Márquez, V.: Monotone vector fields and the proximal point algorithm on Hadamard manifolds. J. Lond. Math. Soc. (2) 79(3), 663–683 (2009)
Li, C., Mordukhovich, B.S., Wang, J., Yao, J.C.: Weak sharp minima on Riemannian manifolds. SIAM J. Optim. 21(4), 1523–1560 (2011)
Luenberger, D.G.: The gradient projection method along geodesics. Management Sci. 18, 620–631 (1972)
Manton, J.H.: A framework for generalising the Newton method and other iterative methods from Euclidean space to manifolds. Numer. Math. 129(1), 91–125 (2015)
Martinet, B.: Régularisation d’inéquations variationnelles par approximations successives. Rev. Française Informat. Recherche Opérationnelle 4(Ser. R-3), 154–158 (1970)
Miller, S.A., Malick, J.: Newton methods for nonsmooth convex minimization: connections among U-Lagrangian, Riemannian Newton and SQP methods. Math. Program. 104(2–3, Ser. B), 609–633 (2005)
Nesterov, Y.E., Todd, M.J.: On the Riemannian geometry defined by self-concordant barriers and interior-point methods. Found. Comput. Math. 2(4), 333–361 (2002)
Pakkaranang, N., Kumam, P., Cho, Y.J.: Proximal point algorithms for solving convex minimization problem and common fixed points problem of asymptotically quasi-nonexpansive mappings in CAT(0) spaces with convergence analysis. Numer. Algorithms 78(3), 827–845 (2018)
Papa Quiroz, E.A., Oliveira, P.R.: Proximal point methods for quasiconvex and convex functions with Bregman distances on Hadamard manifolds. J. Convex Anal. 16(1), 49–69 (2009)
Papa Quiroz, E.A., Quispe, E.M., Oliveira, P.R.: Steepest descent method with a generalized Armijo search for quasiconvex functions on Riemannian manifolds. J. Math. Anal. Appl. 341(1), 467–477 (2008)
Park, F.C., Bobrow, J.E., Ploen, S.R.: A lie group formulation of robot dynamics. Int. J. Rob. Res. 14(6), 609–618 (1995)
Phuengrattana, W., Onjai-uea, N., Cholamjiak, P.: Modified proximal point algorithms for solving constrained minimization and fixed point problems in complete CAT(0) spaces. Mediterr. J. Math. 15(3), 20 (2018). Art. 97
Poljak, B.T.: Subgradient methods: a survey of Soviet research. In: Nonsmooth Optimization (Proceedings of the IIASA Workshop, Laxenburg, 1977), IIASA Proc. Ser., vol. 3, pp. 5–29. Pergamon, Oxford (1978)
Rapcsák, T.: Smooth nonlinear optimization in \(\mathbb R^n\). In: Nonconvex Optimization and Its Applications, vol. 19. Kluwer Academic Publishers, Dordrecht (1997)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control. Optim. 14(5), 877–898 (1976)
Rothaus, O.S.: Domains of positivity. Abh. Math. Sem. Univ. Hamburg 24, 189–235 (1960)
Said, S., Bombrun, L., Berthoumieu, Y., Manton, J.H.: Riemannian Gaussian distributions on the space of symmetric positive definite matrices. IEEE Trans. Inform. Theory 63(4), 2153–2170 (2017)
Sakai, T.: Riemannian geometry. In: Translations of Mathematical Monographs, vol. 149. American Mathematical Society, Providence (1996). Translated from the 1992 Japanese original by the author
Sato, H.: A Dai-Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions. Comput. Optim. Appl. 64(1), 101–118 (2016)
Smith, S.T.: Optimization techniques on Riemannian manifolds. In: Hamiltonian and Gradient Flows, Algorithms and Control, Fields Institute Communications, vol. 3, pp. 113–136. American Mathematical Society, Providence (1994)
Souza, J.C.O., Oliveira, P.R.: A proximal point algorithm for DC fuctions on Hadamard manifolds. J. Global Optim. 63(4), 797–810 (2015)
Sra, S., Hosseini, R.: Conic geometric optimization on the manifold of positive definite matrices. SIAM J. Optim. 25(1), 713–739 (2015)
Udrişte, C.: Convex functions and optimization methods on Riemannian manifolds. In: Mathematics and Its Applications, vol. 297. Kluwer Academic Publishers, Dordrecht (1994)
Ugwunnadi, G.C., Khan, A.R., Abbas, M.: A hybrid proximal point algorithm for finding minimizers and fixed points in CAT(0) spaces. J. Fixed Point Theory Appl. 20(2), 19 (2018). Art. 82
Wang, J.H.: Convergence of Newton’s method for sections on Riemannian manifolds. J. Optim. Theory Appl. 148(1), 125–145 (2011)
Wang, X.M.: Subgradient algorithms on riemannian manifolds of lower bounded curvatures. Optimization 67(1), 179–194 (2018)
Wang, J.H., López, G., Martín-Márquez, V., Li, C.: Monotone and accretive vector fields on Riemannian manifolds. J. Optim. Theory Appl. 146(3), 691–708 (2010)
Wang, X., Li, C., Wang, J., Yao, J.C.: Linear convergence of subgradient algorithm for convex feasibility on Riemannian manifolds. SIAM J. Optim. 25(4), 2334–2358 (2015)
Wang, J., Li, C., Lopez, G., Yao, J.C.: Proximal point algorithms on Hadamard manifolds: linear convergence and finite termination. SIAM J. Optim. 26(4), 2696–2729 (2016)
Wang, X., López, G., Li, C., Yao, J.C.: Equilibrium problems on Riemannian manifolds with applications. J. Math. Anal. Appl. 473(2), 866–891 (2019)
Weber, M., Sra, S.: Riemannian frank-wolfe with application to the geometric mean of positive definite matrices. ArXiv e-prints, pp. 1–21 (2018)
Weinmann, A., Demaret, L., Storath, M.: Total variation regularization for manifold-valued data. SIAM J. Imaging Sci. 7(4), 2226–2257 (2014).
Weinmann, A., Demaret, L., Storath, M.: Mumford-Shah and Potts regularization for manifold-valued data. J. Math. Imaging Vision 55(3), 428–445 (2016).
Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142(1–2, Ser. A), 397–434 (2013)
Wilson, B., Leimeister, M.: Gradient descent in hyperbolic space, pp. 1–10 (2018). arXiv e-prints
Yao, T.T., Bai, Z.J., Zhao, Z.: A Riemannian variant of the Fletcher-Reeves conjugate gradient method for stochastic inverse eigenvalue problems with partial eigendata. Numer. Linear Algebra Appl. 26(2), e2221, 19 (2019)
Zhang, T.: A majorization-minimization algorithm for computing the Karcher mean of positive definite matrices. SIAM J. Matrix Anal. Appl. 38(2), 387–400 (2017)
Zhang, P., Bao, G.: An incremental subgradient method on Riemannian manifolds. J. Optim. Theory Appl. 176(3), 711–727 (2018)
Zhang, H., Sra, S.: First-order methods for geodesically convex optimization. JMLR Workshop Conf. Proc. 49(1), 1–21 (2016)
Zhang, H., Reddi, S.J., Sra, S.: Riemannian SVRG: fast stochastic optimization on Riemannian manifolds. ArXiv e-prints, pp. 1–17 (2016)
Zhu, X.: A Riemannian conjugate gradient method for optimization on the Stiefel manifold. Comput. Optim. Appl. 67(1), 73–110 (2017)
Acknowledgements
This work was supported by CNPq 302473∕2017 − 3, 408151∕2016 − 1 and 424860∕2018 − 0, FAPEG/CNPq/PPP—03/2015 and FAPEG/PRONEM—201710267000532. This work is part of a measure which is co-financed by tax revenue based on the budget approved by the members of the Saxon state parliament. Financial support is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Ferreira, O.P., Louzeiro, M.S., Prudente, L.F. (2020). First Order Methods for Optimization on Riemannian Manifolds. In: Grohs, P., Holler, M., Weinmann, A. (eds) Handbook of Variational Methods for Nonlinear Geometric Data. Springer, Cham. https://doi.org/10.1007/978-3-030-31351-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-31351-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31350-0
Online ISBN: 978-3-030-31351-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)