Skip to main content
SpringerLink
Log in
Book cover

Joint DAGM (German Association for Pattern Recognition) and OAGM Symposium

DAGM/OAGM 2012: Pattern Recognition pp 226–235Cite as

Trust-Region Algorithm for Nonnegative Matrix Factorization with Alpha- and Beta-divergences

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 7476))

Abstract

Nonnegative Matrix Factorization (NMF) is a dimensionality reduction method for representing nonnegative data in a low-dimensional nonnegative space. NMF problems are usually solved with an alternating minimization of a given objective function, using nonnegativity constrained optimization algorithms. This paper is concerned with the projected trust-region algorithm that is adapted to minimize a family of divergences or statistical distances, such as α- or β-divergences that are efficient for solving NMF problems. Using the Cauchy point estimate for the quadratic approximation model, a radius of the trust-region can be estimated efficiently for a symmetric and block-diagonal structure of the corresponding Hessian matrices. The experiments demonstrate a high efficiency of the proposed approach.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bardsley, J.M.: A nonnegatively constrained trust region algorithm for the restoration of images with an unknown blur. Electronic Transactions on Numerical Analysis 20, 139–153 (2005)

    MathSciNet  MATH  Google Scholar 

  2. Berry, M., Browne, M., Langville, A.N., Pauca, P., Plemmons, R.J.: Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics and Data Analysis 52(1), 155–173 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. Cichocki, A., Zdunek, R.: NMFLAB for Signal and Image Processing. Tech. rep., Laboratory for Advanced Brain Signal Processing, BSI, RIKEN, Saitama, Japan (2006), http://www.bsp.brain.riken.jp

  4. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.I.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. Wiley and Sons (2009)

    Google Scholar 

  5. Févotte, C., Bertin, N., Durrieu, J.L.: Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis. Neural Computation 21(3), 793–830 (2009)

    Article  MATH  Google Scholar 

  6. Févotte, C., Idier, J.: Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation 13(3), 1–24 (2010)

    Google Scholar 

  7. Guillamet, D., Vitrià, J., Schiele, B.: Introducing a weighted nonnegative matrix factorization for image classification. Pattern Recognition Letters 24(14), 2447–2454 (2003)

    Article  MATH  Google Scholar 

  8. Heiler, M., Schnoerr, C.: Learning sparse representations by non-negative matrix factorization and sequential cone programming. Journal of Machine Learning Research 7, 1385–1407 (2006)

    MATH  Google Scholar 

  9. Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  10. Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learning spatially localized, parts-based representation. In: Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), vol. 1, pp. I–207–I–212 (2001)

    Google Scholar 

  11. Lin, C.J.: Projected gradient methods for non-negative matrix factorization. Neural Computation 19(10), 2756–2779 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  12. Mauthner, T., Roth, P.M., Bischof, H.: Instant Action Recognition. In: Salberg, A.-B., Hardeberg, J.Y., Jenssen, R. (eds.) SCIA 2009. LNCS, vol. 5575, pp. 1–10. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  13. Minka, T.: Divergence measures and message passing. Tech. Rep. MSR-TR-2005-173, Microsoft Research (2005)

    Google Scholar 

  14. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer Series in Operations Research. Springer, New York (1999)

    Book  MATH  Google Scholar 

  15. Qin, L., Zheng, Q., Jiang, S., Huang, Q., Gao, W.: Unsupervised texture classification: Automatically discover and classify texture patterns. Image and Vision Computing 26(5), 647–656 (2008)

    Article  Google Scholar 

  16. Rojas, M., Steihaug, T.: An interior-point trust-region-based method for large-scale non-negative regularization. Inverse Problems 18, 1291–1307 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. Wang, F.Y., Chi, C.Y., Chan, T.H., Wang, Y.: Nonnegative least-correlated component analysis for separation of dependent sources by volume maximization. IEEE Transactions Pattern Analysis and Machine Intelligence 32(5), 875–888 (2010)

    Article  Google Scholar 

  18. Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with constrained second-order optimization. Signal Processing 87, 1904–1916 (2007)

    Article  MATH  Google Scholar 

  19. Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with quadratic programming. Neurocomputing 71(10-12), 2309–2320 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Telecommunications, Teleinformatics and Acoustics, Wroclaw University of Technology, Wybrzeze Wyspianskiego 27, 50-370, Wroclaw, Poland

    Rafał Zdunek

Authors
  1. Rafał Zdunek
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Electrical Measurement and Measurement Signal Processing, Graz University of Technology, Kronesgasse 5, 8010, Graz, Austria

    Axel Pinz

  2. Institute for Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010, Graz, Austria

    Thomas Pock , Horst Bischof  & Franz Leberl ,  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zdunek, R. (2012). Trust-Region Algorithm for Nonnegative Matrix Factorization with Alpha- and Beta-divergences. In: Pinz, A., Pock, T., Bischof, H., Leberl, F. (eds) Pattern Recognition. DAGM/OAGM 2012. Lecture Notes in Computer Science, vol 7476. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32717-9_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-32717-9_23

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-32716-2

  • Online ISBN: 978-3-642-32717-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation