Skip to main content
SpringerLink
Log in

A DC Programming Approach for Sparse Estimation of a Covariance Matrix

  • Conference paper
Modelling, Computation and Optimization in Information Systems and Management Sciences

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 359))

  • 1584 Accesses

Abstract

We suggest a novel approach to the sparse covariance matrix estimation (SCME) problem using the ℓ1-norm. The resulting optimization problem is nonconvex and very hard to solve. Fortunately, it can be reformulated as DC (Difference of Convex functions) programs to which DC programming and DC Algorithms can be investigated. The main contribution of this paper is to propose a more suitable DC decomposition for solving the SCME problem. The experimental results on both simulated datasets and two real datasets in classification problem illustrate the efficiency of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bickel, P.J., Levina, E.: Some theory for Fisher’s linear discriminant function, naive Bayes, and some alternatives when there are many more variables than observations. Bernoulli 10(6), 989–1010 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  2. Bickel, P.J., Levina, E.: Regularized estimation of large covariance matrices. The Annals of Statistichs 36, 199–227 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bien, J., Tibshirani, R.: Sparse estimation of a covariance matrix. Biometrika 98(4), 807–820 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  4. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundat. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  5. Cai, T., Zhang, C., Zou, H.: Optimal rates of convergence for covariance matrix estimation. The Annals of Statistic 38, 2118–2144 (2010)

    Article  MATH  Google Scholar 

  6. Chaudhuri, S., Drton, M., Richardson, T.S.: Estimation of a covariance matrix with zeros. Biometrika 94, 199–216 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  7. Fawzi, A., Davies, M., Frossard, P.: Dictionary learning for fast classification based on soft-thresholding. International Journal of Computer Vision (2014), http://arxiv.org/abs/1402.1973

  8. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, 2nd edn. Springer, New York (2009)

    Book  MATH  Google Scholar 

  9. Huang, J.Z., Liu, N., Pourahmadi, M., Liu, L.: Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93, 85–98 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  10. Khare, K., Rajaratnam, B.: Wishart distributions for decomposable covariance graph models. Ann. Statist. 39, 514–555 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  11. Lam, C., Fan, J.: Sparsistency and rates of convergence in large covariance matrix estimation. The Annals of Statistics 37, 4254–4278 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  12. Le Thi, H.A., Huynh Van, N., Pham Dinh, T.: Exact penalty and error bounds in DC programming. Journal of Global Optimization 52(3), 509–535 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  13. Le Thi, H.A., Le Hoai, M., Nguyen, V.V., Pham Dinh, T.: A DC Programming approach for feature selection in support vector machines learning. Journal of Advances in Data Analysis and Classification 2(3), 259–278 (2008)

    Article  MATH  Google Scholar 

  14. Le Thi, H.A., Le Hoai, M., Pham Dinh, T.: Optimization based DC programming and DCA for hierarchical clustering. European Journal of Operational Research 183, 1067–1085 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  15. Le Thi, H.A., Le Hoai, M., Pham Dinh, T.: Feature selection in machine learning: An exact penalty approachusing a difference of convex function algorithm. Machine Learning (2014), (published online July 04, 2014), doi:10.1007/s10994-014-5455-y

    Google Scholar 

  16. Le Thi, H.A., Nguyen, M.C.: Self-organizing maps by difference of convex functions optimization. Data Mining and Knowledge Discovery 28, 1336–1365 (2014)

    Article  MathSciNet  Google Scholar 

  17. Le Thi, H.A., Pham Dinh, T.: Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. Journal of Global Optimization 11, 253–285 (1997)

    Article  MATH  Google Scholar 

  18. Le Thi, H.A., Pham Dinh, T.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Annals of Operations Research 133, 23–46 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  19. Le Thi, H.A., Pham Dinh, T., Le Hoai, M., Vo Xuan, T.: DC approximation approaches for sparse optimization. To appear in European Journal of Operational Research (2014)

    Google Scholar 

  20. Le Thi, H.A., Vo Xuan, T., Pham Dinh, T.: Feature selection for linear SVMs under uncertain data: robust optimization based on difference of convex functions Algorithms. Neural Networks 59, 36–50 (2014)

    Article  Google Scholar 

  21. Levina, E., Rothman, A., Zhu, J.: Sparse estimation of large covariance matrices via a nested lasso penalty. Ann. Appl. Stat. 2(1), 245–263 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  22. Liu, H., Wang, L., Zhao, T.: Sparse covariance matrix estimation with eigenvalue contraints. Journal of Computational and Graphical Statistics 23(2), 439–459 (2014)

    Article  MathSciNet  Google Scholar 

  23. Liu, Y., Shen, X.: Multicategory ψ-Learning. Journal of the American Statistical Association 101, 500–509 (2006)

    Article  MathSciNet  Google Scholar 

  24. Liu, Y., Shen, X., Doss, H.: Multicategory ψ-Learning and Support Vector Machine: Computational Tools. Journal of Computational and Graphical Statistics 14, 219–236 (2005)

    Article  MathSciNet  Google Scholar 

  25. Mardia, K.V., Kent, J.T., Bibby, J.M.: Multivariate Analysis. Academic (1979)

    Google Scholar 

  26. Pham Dinh, T., Le Thi, H.A.: Convex analysis approach to D.C. programming: Theory, algorithms and applications. Acta Mathematica Vietnamica 22(1), 289–355 (1997)

    MATH  MathSciNet  Google Scholar 

  27. Pham Dinh, T., Le Thi, H.A.: DC optimization algorithm for solving the trust-region subproblem. SIAM Journal of Optimization 8(1), 476–505 (1998)

    MATH  Google Scholar 

  28. Rothman, A.J., Levina, E., Zhu, J.: Generalized thresholding of large covariance matrices. J. Am. Statist. Assoc. 104, 177–186 (2009)

    Article  MathSciNet  Google Scholar 

  29. Rothman, A.J., Levina, E., Zhu, J.: A new approach to Cholesky-based covariance regularization in high dimensions. Biometrika 97, 539–550 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  30. Thiao, M., Pham Dinh, T., Le Thi, H.A.: DC Programming. In: Le Thi, H.A., Bouvry, P., Pham Dinh, T. (eds.) MCO 2008. CCIS, vol. 14, pp. 348–357. Springer, Heidelberg (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory of Theoretical and Applied Computer Science EA 3097, University of Lorraine, Ile de Saulcy, 57045, Metz, France

    Duy Nhat Phan & Hoai An Le Thi

  2. Laboratory of Mathematics, INSA-Rouen, University of Normandie, 76801, Saint-Etienne-du-Rouvray cedex, France

    Tao Pham Dinh

Authors
  1. Duy Nhat Phan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hoai An Le Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Pham Dinh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Duy Nhat Phan .

Editor information

Editors and Affiliations

  1. Laboratory of Theoretical and Applied Computer Science, University of Lorraine-Metz, Metz, France

    Hoai An Le Thi

  2. Laboratory of Mathematics, National Institute for Applied Sciences-Rouen, Rouen, France

    Tao Pham Dinh

  3. Division of Knowledge Management Systems Institute of Informatics (I-32), Wroclaw University of Technology, Wroclaw, Poland

    Ngoc Thanh Nguyen

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Phan, D.N., Le Thi, H.A., Dinh, T.P. (2015). A DC Programming Approach for Sparse Estimation of a Covariance Matrix. In: Le Thi, H., Pham Dinh, T., Nguyen, N. (eds) Modelling, Computation and Optimization in Information Systems and Management Sciences. Advances in Intelligent Systems and Computing, vol 359. Springer, Cham. https://doi.org/10.1007/978-3-319-18161-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-18161-5_12

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-18160-8

  • Online ISBN: 978-3-319-18161-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation