Skip to main content

Pairwise Clustering with t-PLSI

  • Conference paper
  • 2955 Accesses

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7553)

Abstract

In the past decade, Probabilistic Latent Semantic Indexing (PLSI) has become an important modeling technique, widely used in clustering or graph partitioning analysis. However, the original PLSI is designed for multinomial data and may not handle other data types. To overcome this restriction, we generalize PLSI to t-exponential family based on a recently proposed information criterion called t-divergence. The t-divergence enjoys more flexibility than KL-divergence in PLSI such that it can accommodate more types of noise in data. To optimize the generalized learning objective, we propose a Majorization-Minimization algorithm which multiplicatively updates the factorizing matrices. The new method is verified in pairwise clustering tasks. Experimental results on real-world datasets show that PLSI with t-divergence can improve clustering performance in purity for certain datasets.

Keywords

  • clustering
  • divergence
  • approximation
  • multiplicative update

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-33266-1_51
  • Chapter length: 8 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-33266-1
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arora, R., Gupta, M., Kapila, A., Fazel, M.: Clustering by left-stochastic matrix factorization. In: International Conference on Machine Learning (ICML), pp. 761–768 (2011)

    Google Scholar 

  2. Choi, H., Choi, S., Katake, A., Choe, Y.: Learning alpha-integration with partially-labeled data. In: Proc. of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 14–19 (2010)

    Google Scholar 

  3. Cichocki, A., Lee, H., Kim, Y.D., Choi, S.: Non-negative matrix factorization with α-divergence. Pattern Recognition Letters 29, 1433–1440 (2008)

    CrossRef  Google Scholar 

  4. Cichocki, A., Zdunek, R., Phan, A.H., Amari, S.: Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis. John Wiley (2009)

    Google Scholar 

  5. Ding, C., Li, T., Peng, W.: On the equivalence between non-negative matrix factorization and probabilistic latent semantic indexing. Computational Statistics and Data Analysis 52(8), 3913–3927 (2008)

    CrossRef  MATH  MathSciNet  Google Scholar 

  6. Ding, C., Li, T., Jordan, M.: Convex and semi-nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(1), 45–55 (2010)

    CrossRef  Google Scholar 

  7. Ding, N., Vishwanathan, S.: t-logistic regression. In: Advances in Neural Information Processing Systems, vol. 23, pp. 514–522 (2010)

    Google Scholar 

  8. Ding, N., Vishwanathan, S., Qi, Y.A.: t-divergence based approximate inference. In: Advances in Neural Information Processing Systems, vol. 24, pp. 1494–1502 (2011)

    Google Scholar 

  9. Févotte, C., Bertin, N., Durrieu, J.L.: Nonnegative matrix factorization with the Itakura-Saito divergence: With application to music analysis. Neural Computation 21(3), 793–830 (2009)

    CrossRef  MATH  Google Scholar 

  10. Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International Conference on Research and Development in Information Retrieval (SIGIR), pp. 50–57. ACM (1999)

    Google Scholar 

  11. Hunter, D.R., Lange, K.: A tutorial on MM algorithms. The American Statistician 58(1), 30–37 (2004)

    CrossRef  MathSciNet  Google Scholar 

  12. Mollah, M., Sultana, N., Minami, M.: Robust extraction of local structures by the minimum of beta-divergence method. Neural Networks 23, 226–238 (2010)

    CrossRef  Google Scholar 

  13. Yang, Z., Oja, E.: Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization. IEEE Transactions on Neural Networks 22(12), 1878–1891 (2011)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Zhang, H., Hao, T., Yang, Z., Oja, E. (2012). Pairwise Clustering with t-PLSI. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33266-1_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-33266-1_51

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33265-4

  • Online ISBN: 978-3-642-33266-1

  • eBook Packages: Computer ScienceComputer Science (R0)