High Performance Computing for Mathematical Optimization Problem

Part of the Mathematics for Industry book series (MFI, volume 5)


The semidefinite programming (SDP) problem is one of the central problems in mathematical optimization. The primal-dual interior-point method (PDIPM) is one of the most powerful algorithms for solving SDP problems, and many research groups have employed it for developing software packages. However, two well-known major bottlenecks, i.e., the generation of the Schur complement matrix (SCM) and its Cholesky factorization, exist in the algorithmic framework of the PDIPM. We have developed a new version of the semidefinite programming algorithm parallel version (SDPARA), which is a parallel implementation on multiple CPUs and GPUs for solving extremely large-scale SDP problems with over a million constraints. SDPARA can automatically extract the unique characteristics from an SDP problem and identify the bottleneck. When the generation of the SCM becomes a bottleneck, SDPARA can attain high scalability using a large quantity of CPU cores and some processor affinity and memory interleaving techniques. SDPARA can also perform parallel Cholesky factorization using thousands of GPUs and techniques for overlapping computation and communication if an SDP problem has over 2 million constraints and Cholesky factorization constitutes a bottleneck. We demonstrate that SDPARA is a high-performance general solver for SDPs in various application fields through numerical experiments conducted on the TSUBAME 2.5 supercomputer, and we solved the largest SDP problem (which has over 2.33 million constraints), thereby creating a new world record. Our implementation also achieved 1.713 PFlops in double precision for large-scale Cholesky factorization using 2,720 CPUs and 4,080 GPUs.


Mathematical optimization Semidefinite programming Interior-point method Parallel computing High performance computing 



This research was supported by the Japan Science and Technology Agency (JST), the Core Research of Evolutionary Science and Technology (CREST), and the TSUBAME 2.0 and 2.5 Supercomputer Grand Challenge Program at the Tokyo Institute of Technology.


  1. 1.
    H. Wolkowicz, R. Saigal, L. Vandenberghe (eds.), Handbook of Semidefinite Programming (International Series in Operations Research and Management Science), (Kluwer Academic Publishers, Massachusetts, 2000)Google Scholar
  2. 2.
    M.F. Anjos, J.B. Lasserre (eds.), Handbook on Semidefinite, Conic and Polynomial Optimization (International Series in Operations Research and Management Science). (Springer Science+Business Media, Dordrecht, 2011)Google Scholar
  3. 3.
    M. Yamashita, K. Fujisawa, M. Fukuda, K. Kobayashi, K. Nakata, M. Nakata, in Latest Developments in the SDPA Family for Solving Large-Scale SDPs, ed. by M.F. Anjos and J.B. Lasserre. Handbook on Semidefinite, Cone and Polynomial Optimization: Theory, Algorithms, Software and Applications. (Springer Press, New York, 2011), pp 687–713 (Chapter 24)Google Scholar
  4. 4.
    K. Fujisawa, M. Kojima, K. Nakata, Exploiting sparsity in primal-dual interior-point methods for semidefinite programming. Math. Prog. 79, 235–253 (1997)MathSciNetMATHGoogle Scholar
  5. 5.
    K. Fujisawa, K. Nakata, M. Yamashita, M. Fukuda, SDPA project: solving large-scale semidefinite programs. J. Oper. Res. Soc. Japan 50, 278–298 (2007)MathSciNetMATHGoogle Scholar
  6. 6.
    B. Borchers, CSDP, a C library for semidefinite programming. Optim. Meth. Softw. 11, 12, 613–623 (1999)Google Scholar
  7. 7.
    J.F. Sturm, SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optim. Meth. Softw. 11, 12, 625–653 (1999)Google Scholar
  8. 8.
    K.-C. Toh, M.J. Todd, R.H. Tütüncü, SDPT3—MATLAB software package for semidefinite programming, version 1.3. Optim. Meth. Softw. 11, 12, 545–581 (1999)Google Scholar
  9. 9.
    M. Yamashita, K. Fujisawa, M. Kojima, SDPARA: semidefinite programming algorithm parallel version. Parallel Comput. 29, 1053–1067 (2003)MathSciNetCrossRefGoogle Scholar
  10. 10.
    M. Yamashita, K. Fujisawa, M. Fukuda, K. Nakata, M. Nakata, Parallel solver for semidefinite programming problem having sparse Schur complement matrix. ACM Trans. Math. Softw. 39(12), (2012)Google Scholar
  11. 11.
    K. Fujisawa, T. Endo, H. Sato, M. Yamashita, S. Matsuoka, M. Nakata, in Proceedings of the 2012 ACM/IEEE Conference on Supercomputing, SC’12. High-Performance General Solver for Extremely Large-Scale Semidefinite Programming Problems (2012)Google Scholar
  12. 12.
    K. Fujisawa, T. Endo, Y. Yasui, H. Sato, N. Matsuzawa, S. Matsuoka, H.Waki, in The 28th IEEE International Parallel and Distributed Processing Symposium (IPDPS2014). Peta-scale General Solver for Semidefinite Programming Problems with over Two Million Constraints (2014)Google Scholar
  13. 13.
    I.D. Ivanov, E. de Klerk, Parallel implementation of a semidefinite programming solver based on CSDP in a distributed memory cluster. Optim. Meth. Softw. 25(3), 405–420 (2010)CrossRefMATHGoogle Scholar
  14. 14.
    S. Benson, Parallel computing on semidefinite programs. (Mathematics and Computer Science Division Argonne Science Laboratory, IL, 2002) (Preprint ANL/MCS-P939-0302)Google Scholar
  15. 15.
    K. Kobayashi, S. Kim, M. Kojima, Correlative sparsity in primal-dual interior-point methods for LP, SDP and SOCP. Appl. Math. Optim. 58, 69–88 (2008)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    P.R. Amestoy, I.S. Duff, J.-Y. L’Excellent, Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput. Methods in Appl. Mech. Eng. 184, 501–520 (2000)CrossRefMATHGoogle Scholar
  17. 17.
    S. Burer, On the copositive representation of binary and continuous nonconvex quadratic programs. Math. Prog. 120, 479–495 (2009)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Q. Zhao, S.E. Karisch, F. Rendl, H. Wolkowicz, Semidefinite programming relaxations for the quadratic assignment problem. J. Comb. Optim. 2, 71–109 (1998)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    P.A.M. Dirac, The quantum theory of the electron. Roy. Soc. A 123, 714 (1929) (London)Google Scholar
  20. 20.
    A.J. Coleman, Structure of fermion density matrices. Rev. Mod. Phys. 35, 668–687 (1963)CrossRefGoogle Scholar
  21. 21.
    C. Garrod, J.K. Percus, Reduction of the \(N\)-particle variational problem. J. Math. Phys. 5, 1756–1776 (1964)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    M. Nakata, H. Nakatsuji, M. Ehara, M. Fukuda, K. Nakata, K. Fujisawa, Variational calculations of fermion second-order reduced density matrices by semidefinite programming algorithm. J. Chem. Phys. 114, 8282–8292 (2001)CrossRefGoogle Scholar
  23. 23.
    Z. Zhao, B.J. Braams, M. Fukuda, M.L. Overton, J.K. Percus, The reduced density matrix method for electronic structure calculations and the role of three-index representability conditions. J. Chem. Phys. 120, 2095–2104 (2004)CrossRefGoogle Scholar
  24. 24.
    F. Alizadeh, J.-P.A. Haeberly, M.L. Overton, Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results. SIAM J. Optim. 8, 746–768 (1998)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    C. Helmberg, F. Rendl, R.J. Vanderbei, H. Wolkowicz, An interior-point method for semidefinite programming. SIAM J. Optim. 6, 342–361 (1996)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    M. Kojima, S. Shindoh, S. Hara, Interior-point methods for the monotone semidefinite linear complementarity problems. SIAM J. Optim. 7, 86–125 (1997)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    R.D.C Monteiro, Primal-dual path following algorithms for semidefinite programming. SIAM J. Optim. 7, 663–678 (1997)Google Scholar
  28. 28.
    Y.E. Nesterov, M.J. Todd, Primal-dual interior-point methods for self-scaled cones. SIAM J. Optim 8, 324–364 (1998)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    S. Matsuoka, T. Endo, N. Maruyama, H. Sato, S. Takizawa: The Total Picture of TSUBAME2.0. TSUBAME e-Science Journal, GSIC, Tokyo Institute of Technology, Vol. 1, 2–4 (2010).Google Scholar
  30. 30.
    T. Endo, A. Nukada, S. Matsuoka, N. Maruyama, in Proceedings of IEEE IPDPS10. Linpack Evaluation on a Supercomputer with Heterogeneous Accelerators, pp. 1–8 (2010)Google Scholar
  31. 31.
    A. Petitet, R.C. Whaley, J. Dongarra, A. Cleary, HPL—A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers,

Copyright information

© Springer Japan 2014

Authors and Affiliations

  1. 1.Institute of Mathematics for IndustryKyushu UniversityFukuokaJapan

Personalised recommendations