Journal of Scientific Computing

, Volume 66, Issue 3, pp 1175–1203 | Cite as

Trace-Penalty Minimization for Large-Scale Eigenspace Computation

  • Zaiwen Wen
  • Chao Yang
  • Xin Liu
  • Yin Zhang


In a block algorithm for computing relatively high-dimensional eigenspaces of large sparse symmetric matrices, the Rayleigh-Ritz (RR) procedure often constitutes a major bottleneck. Although dense eigenvalue calculations for subproblems in RR steps can be parallelized to a certain level, their parallel scalability, which is limited by some inherent sequential steps, is lower than dense matrix-matrix multiplications. The primary motivation of this paper is to develop a methodology that reduces the use of the RR procedure in exchange for matrix-matrix multiplications. We propose an unconstrained trace-penalty minimization model and establish its equivalence to the eigenvalue problem. With a suitably chosen penalty parameter, this model possesses far fewer undesirable full-rank stationary points than the classic trace minimization model. More importantly, it enables us to deploy algorithms that makes heavy use of dense matrix-matrix multiplications. Although the proposed algorithm does not necessarily reduce the total number of arithmetic operations, it leverages highly optimized operations on modern high performance computers to achieve parallel scalability. Numerical results based on a preliminary implementation, parallelized using OpenMP, show that our approach is promising.


Eigenvalue computation Exact quadratic penalty approach Gradient methods 

Mathematics Subject Classification

15A18 65F15 65K05 90C06 



The computational results were obtained at the National Energy Research Scientific Computing Center (NERSC), which is supported by the Director, Office of Advanced Scientific Computing Research of the U.S. Department of Energy under contract number DE-AC02-05CH11232. Z. Wen would like to thank Prof. Michael Ulbrich for hosting his visit at Technische Universität München. X. Liu would like to thank Prof. Yuhong Dai for discussing nonlinear programming techniques for eigenvalue computation. C. Yang would like to thank Dr. Eugene Vencharynski for helping test EigPen, especially the preconditioned version. The authors are grateful to Prof. Chi-Wang Shu, the associate editor and the anonymous referees for their detailed and valuable comments and suggestions.


  1. 1.
    Anderson, E., Bai, Z., Dongarra, J., Greenbaum, A., McKenney, A., Du Croz, J., Hammerling, S., Demmel, J., Bischof, C., Sorensen, D.: Lapack: a portable linear algebra library for high-performance computers, in Proceedings of the 1990 ACM/IEEE conference on Supercomputing, Supercomputing ’90, IEEE Computer Society Press, pp. 2–11 (1990)Google Scholar
  2. 2.
    Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)CrossRefMathSciNetzbMATHGoogle Scholar
  3. 3.
    Blackford, L.S., Choi, J., Cleary, A., D’Azeuedo, E., Demmel, J., Dhillon, I., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK User’s Guide. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (1997)CrossRefGoogle Scholar
  4. 4.
    Courant, R.: Variational methods for the solution of problems of equilibrium and vibrations. Bull. Am. Math. Soc. 49, 1–23 (1943)CrossRefMathSciNetzbMATHGoogle Scholar
  5. 5.
    Dai, Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315–330 (2002)CrossRefMathSciNetzbMATHGoogle Scholar
  6. 6.
    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)CrossRefMathSciNetzbMATHGoogle Scholar
  7. 7.
    Knyazev, A., Argentati, M., Lashuk, I., Ovtchinnikov, E.: Block locally optimal preconditioned eigenvalue xolvers (blopex) in hypre and petsc. SIAM J. Sci. Comput. 29, 2224–2239 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  8. 8.
    Knyazev, Andrew V.: Toward the optimal preconditioned eigensolver: locally optimal block preconditioned conjugate gradient method. SIAM J. Sci. Comput. 23, 517–541 (2001)CrossRefMathSciNetzbMATHGoogle Scholar
  9. 9.
    Kronik, L., Makmal, A., Tiago, M., Alemany, M.M.G., Huang, X., Saad, Y., Chelikowsky, J.R.: PARSEC - the pseudopotential algorithm for real-space electronic structure calculations: recent advances and novel applications to nanostructures. Phys. Status Solidi. (b) 243, 1063–1079 (2006)CrossRefGoogle Scholar
  10. 10.
    Lehoucq, R.B., Sorensen, D.C., Yang, C.: ARPACK users’ guide: Solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods, vol. 6 of software, environments, and tools, society for industrial and applied mathematics (SIAM), Philadelphia, PA, (1998)Google Scholar
  11. 11.
    Nocedal, Jorge, Wright, Stephen J.: Numerical Optimization, Springer Series in Operations Research and Financial Engineering, 2nd edn. Springer, New York (2006)Google Scholar
  12. 12.
    Saad, Yousef, Chelikowsky, James R., Shontz, Suzanne M.: Numerical methods for electronic structure calculations of materials. SIAM Rev. 52, 3–54 (2010)CrossRefMathSciNetzbMATHGoogle Scholar
  13. 13.
    Sameh, Ahmed H., Wisniewski, John A.: A trace minimization algorithm for the generalized eigenvalue problem. SIAM J. Numer. Anal. 19, 1243–1259 (1982)CrossRefMathSciNetzbMATHGoogle Scholar
  14. 14.
    Stathopoulos, Andreas, McCombs, James R.: Nearly optimal preconditioned methods for hermitian eigenproblems under limited memory. Part II: seeking many eigenvalues. SIAM J. Sci. Comput. 29, 2162–2188 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  15. 15.
    Stathopoulos, A., McCombs, J.R.: PRIMME: preconditioned iterative multimethod eigensolver-methods and software description. ACM Trans. Math. Softw. 37, 21:1–21:30 (2010)CrossRefGoogle Scholar
  16. 16.
    Sun, Wenyu, Yuan, Yaxiang: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006)Google Scholar
  17. 17.
    Teter, M.P., Payne, M.C., Allan, D.C.: Solution of schrödinger’s equation for large systems. Phys. Rev. B 40, 12255–12263 (1989)CrossRefGoogle Scholar
  18. 18.
    Yang, Chao, Meza, Juan C., Lee, Byounghak, Wang, Lin-Wang: KSSOLV–a MATLAB toolbox for solving the Kohn-Sham equations. ACM Trans. Math. Softw. 36, 1–35 (2009)CrossRefMathSciNetGoogle Scholar
  19. 19.
    Zhang, Hongchao, Hager, William W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043–1056 (2004)CrossRefMathSciNetzbMATHGoogle Scholar
  20. 20.
    Zhou, Y.: A block Chebyshev-Davidson method with inner-outer restart for large eigenvalue problems. J. Comput. Phys. 229, 9188–9200 (2010)CrossRefMathSciNetzbMATHGoogle Scholar
  21. 21.
    Zhou, Y., Saad, Y.: A Chebyshev-Davidson algorithm for large symmetric eigenproblems. SIAM J. Matrix Anal. Appl. 29, 954–971 (2007)CrossRefMathSciNetzbMATHGoogle Scholar
  22. 22.
    Zhou, Y., Saad, Y.: Block krylovschur method for large symmetric eigenvalue problems. Numer. Algorithms 47, 341–359 (2008)CrossRefMathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Beijing International Center for Mathematical ResearchPeking UniversityBeijingChina
  2. 2.Computational Research DivisionLawrence Berkeley National LaboratoryBerkeleyUSA
  3. 3.State Key Laboratory of Scientific and Engineering Computing, Academy of Mathematics and Systems ScienceChinese Academy of SciencesBeijingChina
  4. 4.Department of Computational and Applied MathematicsRice UniversityHoustonUSA

Personalised recommendations