Optimization Letters

, Volume 13, Issue 8, pp 1773–1791 | Cite as

First order optimality conditions and steepest descent algorithm on orthogonal Stiefel manifolds

  • Petre BirteaEmail author
  • Ioan Caşu
  • Dan Comănescu
Original Paper


Considering orthogonal Stiefel manifolds as constraint manifolds, we give an explicit description of a set of local coordinates that also generate a basis for the tangent space in any point of the orthogonal Stiefel manifolds. We show how this construction depends on the choice of a submatrix of full rank. Embedding a gradient vector field on an orthogonal Stiefel manifold in the ambient space, we give explicit necessary and sufficient conditions for a critical point of a cost function defined on such manifolds. We explicitly describe the steepest descent algorithm on the orthogonal Stiefel manifold using the ambient coordinates and not the local coordinates of the manifold. We point out the dependence of the recurrence sequence that defines the algorithm on the choice of a full rank submatrix. We illustrate the algorithm in the case of Brockett cost functions.


Steepest descent algorithm Optimization Constraint manifold Orthogonal Stiefel manifold Brockett cost function 



This work was supported by a Grant of Ministery of Research and Innovation, CNCS - UEFISCDI, project number PN-III-P4-ID-PCE-2016-0165, within PNCDI III.


  1. 1.
    Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, Oxford (2008)CrossRefGoogle Scholar
  2. 2.
    Adler, R.L., Dedieu, J.-P., Margulies, J.Y., Martens, M., Shub, M.: Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA J. Numer. Anal. 22, 359–390 (2002)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Balogh, J., Csendes, T., Rapcsák, T.: Some global optimization problems on Stiefel manifolds. J. Global Optim. 30, 91–101 (2004)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Birtea, P., Comănescu, D.: Geometrical dissipation for dynamical systems. Commun. Math. Phys. 316, 375–394 (2012)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Birtea, P., Comănescu, D.: Hessian operators on constraint manifolds. J. Nonlinear Sci. 25, 1285–1305 (2015)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Birtea, P., Comănescu, D.: Newton algorithm on constraint manifolds and the 5-electron Thomson problem. J. Optim. Theory Appl. 173, 563–583 (2017)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bolla, M., Michaletzky, G., Tusnády, G., Ziermann, M.: Extrema of sums of heterogeneous quadratic forms. Linear Algebra Appl. 269, 331–365 (1998)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2010)CrossRefGoogle Scholar
  9. 9.
    Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155, 57–79 (2016)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chu, M.T., Trendafilov, N.T.: The orthogonally constrained regression revisited. J. Comput. Graph. Stat. 10, 746–771 (2001)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Eldén, L., Park, H.: A Procrustes problem on the Stiefel manifold. Numer. Math. 82, 599–619 (1999)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Fraikin, C., Hüper, K., Van Dooren, P.: Optimization over the Stiefel manifold. PAMM 7, 1062205–1062206 (2007)CrossRefGoogle Scholar
  14. 14.
    Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: Fitzgibbon, W., Kuznetsov, Y.A., Neittaanmaki, P., Neittaanmaki, O. (eds.) Modeling, Simulation and Optimization for Science and Technology, Computational Methods in Applied Sciences, vol. 34, pp. 59–82. Springer, Dordrecht (2014)Google Scholar
  15. 15.
    Glowinski, R., Marrocco, A.: Sur l’approximation par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Rev. Fr. Automat. Inform. 9, 41–76 (1975)zbMATHGoogle Scholar
  16. 16.
    Jiang, B., Dai, Y.-H.: A framework of constraint preserving update schemes for optimization on Stiefel manifold. Math. Program. 153, 535–575 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Kanamori, T., Takeda, A.: Non-convex optimization on Stiefel manifold and applications to machine learning. In: Proceedings of the 19th international conference on neural information processing, ICONIP 2012, Part I, Doha, pp. 109–116 (2012)CrossRefGoogle Scholar
  18. 18.
    Krakowski, K.A., Machado, L., Leite, F.S., Batista, J.: A modified Casteljau algorithm to solve interpolation problems on Stiefel manifolds. J. Comput. Appl. Math. 311, 84–99 (2017)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Manton, J.H.: Optimization algorithms exploiting unitary constraints. IEEE Trans. Signal Process. 50, 635–650 (2002)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Nishimori, Y., Akaho, S.: Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold. Neurocomputing 67, 106–135 (2005)CrossRefGoogle Scholar
  21. 21.
    Petersen, K.B., Pedersen, M.S.: The Matrix Cookbook. Technical University of Denmark (2012).
  22. 22.
    Rapcsák, T.: On minimization on Stiefel manifolds. Eur. J. Oper. Res. 143, 365–376 (2002)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Rosen, J.B.: The gradient projection method for nonlinear programming. Part II. Nonlinear constraints. J. Soc. Ind. Appl. Math. 9, 514–532 (1961)CrossRefGoogle Scholar
  24. 24.
    Shub, M.: Some remarks on dynamical systems and numerical analysis. In: Dynamical Systems and Partial Differential Equations, pp. 69–91. University Simon Bolivar, Caracas (1986)Google Scholar
  25. 25.
    Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. 142, 397–434 (2013)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Zhang, Y.: Recent advances in alternating direction methods: theory and practice. In: IPAM workshop: numerical methods for continuous optimization. UCLA, Los Angeles (2010)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of MathematicsWest University of TimişoaraTimişoaraRomania

Personalised recommendations