Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

A generalized power iteration method for solving quadratic problem on the Stiefel manifold

Abstract

In this paper, we first propose a novel generalized power iteration (GPI) method to solve the quadratic problem on the Stiefel manifold (QPSM) as \(min_{W^{T}W=I}\) Tr(W T AW − 2W T B) along with the theoretical analysis. Accordingly, its special case known as the orthogonal least square regression (OLSR) is under further investigation. Based on the aforementioned studies, we then majorly focus on solving the unbalanced orthogonal procrustes problem (UOPP). As a result, not only a general convergent algorithm is derived theoretically but the efficiency of the proposed approach is verified empirically as well.

This is a preview of subscription content, log in to check access.

References

  1. 1

    Green B. The orthogonal approximation of an oblique simple structure in factor analysis. Psychometrika, 1952, 17: 429–440

  2. 2

    Hurley J, Cattell R. The procrustes program: producing direct rotation to test a hypothesized factor structure. Behav Sci, 1962, 6: 258–262

  3. 3

    Golub G H, van Loan C F. Matrix Computations. Baltimore: The Johns Hopkins University Press, 1989

  4. 4

    Thomas V. Algorithms for the weighted orthogonal procrustes problem and other least squares problems. Dissertation for Ph.D. Degree. Umeå: Umeå University, 2006

  5. 5

    Souza P, Leite C, Borges H, et al. Online algorithm based on support vectors for orthogonal regression. Pattern Recogn Lett, 2013, 34: 1394–1404

  6. 6

    Chu M, Trendafilov N. The orthogonally constrained regression revisted. J Comput Graph Stat, 2001, 10: 746–771

  7. 7

    Green B, Goers J. A problem with congruence. In: Proceedings of the Annual Meeting of the Psychometric Society, Monterey, 1979

  8. 8

    Park H. A parallel algorithm for the unbalanced orthogonal procrustes problem. Parall Comput, 1991, 17: 913–923

  9. 9

    Bojanczyk A, Lutoborski A. The procrustes problem for orthogonal stiefel matrices. SIAM J Sci Comput, 1999, 21: 1291–1304

  10. 10

    Zhang Z Y, Du K Q. Successive projection method for solving the unbalanced procrustes problem. Sci China Ser A-Math, 2006, 49: 971–986

  11. 11

    Zhang Z Y, Huang Y S. A projection method for least square problems with quadratic equality constraint. SIAM J Matr Anal Appl, 2003, 25: 188–212

  12. 12

    Xia Y, Han Y W. Partial lagrangian relaxation for the unbalanced orthogonal procrustes problem. Math Meth Oper Res, 2014, 79: 225–237

  13. 13

    Journée M, Nesterov Y, Richtárik P, et al. Generalized power method for sparse principal component analysis. J Mach Learn Res, 2008, 11: 517–553

  14. 14

    Fiori S. Formulation and integration of learning differential equations on the Stiefel manifold. IEEE Trans Neural Netw, 2005, 16: 1697–1701

  15. 15

    Nie F P, Yuan J J, Huang H. Optimal mean robust principal component analysis. In: Proceedings of the 31st International Conference on Machine Learning, Beijing, 2014. 2755–2763

  16. 16

    Kaneko T, Fiori S, Tanaka T. Empirical arithmetic averaging over the compact stiefel manifold. IEEE Trans Signal Process, 2013, 61: 883–894

  17. 17

    Gander W, Golub G H, von Matt U. A constrained eigenvalue problem. In: Numerical Linear Algebra, Digital Signal Processing and Parallel Algorithms. Berlin: Springer, 1991. 677–686

  18. 18

    Joachims T. Transductive learning via spectral graph partitioning. In: Proceedings of the 20th International Conference on Machine Learning, Washington, 2003. 51–60

Download references

Author information

Correspondence to Feiping Nie or Rui Zhang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nie, F., Zhang, R. & Li, X. A generalized power iteration method for solving quadratic problem on the Stiefel manifold. Sci. China Inf. Sci. 60, 112101 (2017). https://doi.org/10.1007/s11432-016-9021-9

Download citation

Keywords

  • quadratic problem
  • Stiefel manifold
  • power iteration
  • procrustes problem
  • orthogonal least square regression