Advertisement

Topic Grouper: An Agglomerative Clustering Approach to Topic Modeling

  • Daniel PfeiferEmail author
  • Jochen L. LeidnerEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11437)

Abstract

We introduce Topic Grouper as a complementary approach in the field of probabilistic topic modeling. Topic Grouper creates a disjunctive partitioning of the training vocabulary in a stepwise manner such that resulting partitions represent topics. Topic generation is based on a simple probabilistic model and agglomerative clustering, where clusters are formed as sets of words from the vocabulary. The resulting binary tree of topics may act as a containment hierarchy typically with more general topics towards the root of tree and more specific topics towards the leaves. As opposed to other topic modeling approaches, Topic Grouper avoids the need for hyper parameter optimizations.

As part of an evaluation, we show that Topic Grouper has reasonable predictive power but also a reasonable complexity. It can deal well with stop words and function words. Also, it can handle topic distributions, where some topics are more frequent than others. We present examples of computed topics which appear as conclusive and coherent.

Keywords

Topic modeling Topic analysis Clustering Probabilistic topic models Text collection browsing Exploratory data analysis 

References

  1. 1.
    Aldous, D.J.: Exchangeability and related topics. In: Hennequin, P.L. (ed.) École d’Été de Probabilités de Saint-Flour XIII — 1983. LNM, vol. 1117, pp. 1–198. Springer, Heidelberg (1985).  https://doi.org/10.1007/BFb0099421CrossRefGoogle Scholar
  2. 2.
    Asuncion, A., Welling, M., Smyth, P., Teh, Y.W.: On smoothing and inference for topic models. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2009, pp. 27–34. AUAI Press, Arlington (2009)Google Scholar
  3. 3.
    Blei, D.M., Jordan, M.I., Griffiths, T.L., Tenenbaum, J.B.: Hierarchical topic models and the nested Chinese restaurant process. In: Proceedings of the 16th International Conference on Neural Information Processing Systems, NIPS 2003, pp. 17–24. MIT Press, Cambridge (2003)Google Scholar
  4. 4.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  5. 5.
    Buntine, W.: Estimating likelihoods for topic models. In: Zhou, Z.-H., Washio, T. (eds.) ACML 2009. LNCS (LNAI), vol. 5828, pp. 51–64. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-05224-8_6CrossRefGoogle Scholar
  6. 6.
    Buntine, W., Jakulin, A.: Discrete component analysis. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds.) SLSFS 2005. LNCS, vol. 3940, pp. 1–33. Springer, Heidelberg (2006).  https://doi.org/10.1007/11752790_1CrossRefGoogle Scholar
  7. 7.
    Chang, J., Boyd-Graber, J.L., Gerrish, S., Wang, C., Blei, D.M.: Reading tea leaves: How humans interpret topic models. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems, 7–10 December 2009, Vancouver, British Columbia, Canada, pp. 288–296. Curran Associates, Inc. (2009)Google Scholar
  8. 8.
    Chen, D., Sain, S.L., Guo, K.: Data mining for the online retail industry: a case study of RFM model-based customer segmentation using data mining. J. Database Mark. Customer Strategy Manag. 19(3), 197–208 (2012)CrossRefGoogle Scholar
  9. 9.
    Chen, M.C., Lin, C.P.: A data mining approach to product assortment and shelf space allocation. Expert Syst. Appl. 32(4), 976–986 (2007)CrossRefGoogle Scholar
  10. 10.
    Griffiths, T.L., Steyvers, M.: Finding scientific topics. Proc. Nat. Acad. Sci. 101(Suppl. 1), 5228–5235 (2004)CrossRefGoogle Scholar
  11. 11.
    Hofmann, T.: The cluster-abstraction model: unsupervised learning of topic hierarchies from text data. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, IJCAI 1999, vol. 2, pp. 682–687. Morgan Kaufmann, San Francisco (1999)Google Scholar
  12. 12.
    Hofmann, T.: Probabilistic latent semantic analysis. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, UAI 1999, pp. 289–296. Morgan Kaufmann, San Francisco (1999)Google Scholar
  13. 13.
    Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice-Hall Inc., Upper Saddle River (1988)zbMATHGoogle Scholar
  14. 14.
    Kamvar, S.D., Klein, D., Manning, C.D.: Interpreting and extending classical agglomerative clustering algorithms using a model-based approach. In: Proceedings of the Nineteenth International Conference on Machine Learning, ICML, pp. 283–290. Morgan Kaufmann, San Francisco (2002)Google Scholar
  15. 15.
    Kim, J.H., Kim, D., Kim, S., Oh, A.: Modeling topic hierarchies with the recursive Chinese restaurant process. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM 2012, pp. 783–792. ACM, New York (2012)Google Scholar
  16. 16.
    Lance, G., Williams, W.: A generalized sorting strategy for computer classifications. Nature 212, 218 (1966)CrossRefGoogle Scholar
  17. 17.
    Lance, G., Williams, W.: A general theory of classificatory sorting strategies. I. hierarchical systems. Comput. J. 9, 373–380 (1967)CrossRefGoogle Scholar
  18. 18.
    Lau, J.H., Newman, D., Baldwin, T.: Machine reading tea leaves: automatically evaluating topic coherence and topic model quality. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2014, pp. 530–539. Association for Computational Linguistics, Gothenburg (2014)Google Scholar
  19. 19.
    Li, W., McCallum, A.: Pachinko allocation: DAG-structured mixture models of topic correlations. In: Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, ICML 2006, PA, USA. ACM, New York (2006)Google Scholar
  20. 20.
    Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, New York (2008)CrossRefGoogle Scholar
  21. 21.
    Murtagh, F.: A survey of recent advances in hierarchical clustering algorithms. Comput. J. 26(4), 354–359 (1983)CrossRefGoogle Scholar
  22. 22.
    Newman, D., Lau, J.H., Grieser, K., Baldwin, T.: Automatic evaluation of topic coherence. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT, pp. 100–108. Association for Computational Linguistics, Stroudsburg (2010)Google Scholar
  23. 23.
    Paisley, J., Wang, C., Blei, D.M., Jordan, M.I.: Nested hierarchical Dirichlet processes. IEEE Trans. Pattern Anal. Mach. Intell. 37(2), 256–270 (2015)CrossRefGoogle Scholar
  24. 24.
    Tan, Y., Ou, Z.: Topic-weak-correlated latent Dirichlet allocation. In: 7th International Symposium on Chinese Spoken Language Processing, pp. 224–228 (2010)Google Scholar
  25. 25.
    Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Sharing clusters among related groups: hierarchical Dirichlet processes. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 17, NIPS, pp. 1385–1392. MIT Press (2005)Google Scholar
  26. 26.
    Vaithyanathan, S., Dom, B.: Model-based hierarchical clustering. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, UAI, pp. 599–608. Morgan Kaufmann, San Francisco (2000)Google Scholar
  27. 27.
    Wallach, H.M., Mimno, D.M., McCallum, A.: Rethinking LDA: Why priors matter. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) NIPS, pp. 1973–1981. Curran Associates, Inc. (2009)Google Scholar
  28. 28.
    Wallach, H.M., Murray, I., Salakhutdinov, R., Mimno, D.: Evaluation methods for topic models. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 1105–1112. ACM, New York (2009)Google Scholar
  29. 29.
    Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2011, pp. 448–456. ACM, New York (2011)Google Scholar
  30. 30.
    Ward Jr., J.H.: Hierarchical grouping to optimize an objective function. J. Am. Stat. Assoc. 58(301), 236–244 (1963)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Xu, R., Wunsch, D.I.: Survey of clustering algorithms. IEEE Trans. Neural Networks 16(3), 645–678 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Medical InformaticsHeilbronn University of Applied SciencesHeilbronnGermany
  2. 2.Refinitiv LabsLondonUK
  3. 3.University of SheffieldSheffieldUK

Personalised recommendations