Advertisement

Frontiers of Computer Science

, Volume 11, Issue 5, pp 786–802 | Cite as

Topic evolution based on the probabilistic topic model: a review

  • Houkui Zhou
  • Huimin YuEmail author
  • Roland Hu
Review Article
  • 224 Downloads

Abstract

Accurately representing the quantity and characteristics of users’ interest in certain topics is an important problem facing topic evolution researchers, particularly as it applies to modern online environments. Search engines can provide information retrieval for a specified topic from archived data, but fail to reflect changes in interest toward the topic over time in a structured way. This paper reviews notable research on topic evolution based on the probabilistic topic model from multiple aspects over the past decade. First, we introduce notations, terminology, and the basic topic model explored in the survey, then we summarize three categories of topic evolution based on the probabilistic topic model: the discrete time topic evolution model, the continuous time topic evolutionmodel, and the online topic evolution model. Next, we describe applications of the topic evolution model and attempt to summarize model generalization performance evaluation and topic evolution evaluation methods, as well as providing comparative experimental results for different models. To conclude the review, we pose some open questions and discuss possible future research directions.

Keywords

topic evolution probabilistic topic models text corpora evaluation method 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

The authors would like to thank the anonymous reviewers for their constructive comments and suggestions, which significantly contributed to improving the manuscript. This work was supported by the National Key Basic Research Project of China (973 Program) (2012CB316400), the National Nature Science Foundation of China (Grant Nos. 61471321, 61202400, 31300539, and 31570629), the Zhejiang Provincial Natural Science Foundation of China (LY15C140005, LY16F010004), Science and Technology Department of Zhejiang Province Public Welfare Project (2016C31G2010057, 2015C31004), Fundamental Research Funds for the Central Universities (172210261) and the Zhejiang Provincial Key Laboratory of Forestry Intelligent Monitoring and Information Technology Research.

Supplementary material

11704_2016_5442_MOESM1_ESM.ppt (130 kb)
Supplementary material, approximately 130 KB.

References

  1. 1.
    Allan J. Introduction to topic detection and tracking. Topic Detection And Tracking. The Information Retrieval Series, Vol 12. Springer US, 2002, 1–16CrossRefGoogle Scholar
  2. 2.
    Allan J, Carbonell J G, Doddington G, Yamron J, Yang Y. Topic detection and tracking pilot study final report. In: Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop. 1998, 194–218Google Scholar
  3. 3.
    Nallapati R, Feng A, Peng F, Allan J. Event threading within news topics. In: Proceedings of the 13th ACM International Conference on Information and Knowledge Management. 2004, 446–453Google Scholar
  4. 4.
    Morinaga S, Yamanishi K. Tracking dynamics of topic trends using a finite mixture model. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004, 811–816Google Scholar
  5. 5.
    Kumar R, Mahadevan U, Sivakumar D. A graph-theoretic approach to extract storylines from search results. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004, 216–225Google Scholar
  6. 6.
    Lin F R, Huang F M, Liang C H. Individualized storyline-based news topic retrospection. In: Proceedings of Pacific Asia Conference on Information Systems: Managing Diversity in Digital Enterprises. 2007Google Scholar
  7. 7.
    Ahmed A, Ho Q, Teo C H, Eisenstein J, Smola A J, Xing E P. Online inference for the infinite topic-cluster model: storylines from streaming text. In: Proceedings of the International Conference on Artificial Intelligence and Statistics. 2011, 101–109Google Scholar
  8. 8.
    Hofmann T. Unsupervised learning by probabilistic latent semantic analysis. Machine Learning, 2001, 42(1): 177–196MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Blei D M, Ng A Y, Jordan M I. Latent dirichlet allocation. Journal of Machine Learning Research, 2003, 3: 993–1022zbMATHGoogle Scholar
  10. 10.
    Shan B, Li F. A survey of topic evolution based on LDA. Journal of Chinese Information Processing, 2010, 24(1): 43–49Google Scholar
  11. 11.
    Elshamy W. Continuous-time infinite dynamic topic models. Dissertation for the Doctoral Degree. Manhattan: Kansas State University, 2013Google Scholar
  12. 12.
    Daud A, Li J Z, Zhou L Z, Muhammad F. Knowledge discovery through directed probabilistic topic models: a survey. Frontiers of Computer Science in China, 2010, 4(2): 280–301CrossRefGoogle Scholar
  13. 13.
    Steyvers M, Griffiths T. Probabilistic topic models. Handbook of Latent Semantic Analysis, 2007, 427(2): 424–440Google Scholar
  14. 14.
    Blei D M, Lafferty J D. Dynamic topic models. In: Proceedings of the 23rd ACM International Conference on Machine Learning. 2006, 113–120Google Scholar
  15. 15.
    Blei D M, Lafferty J D. A correlated topic model of science. Annals of Applied Statistics, 2007, 1(1): 17–35MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Blei DM, Griffiths T L, Jordan M I. The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. Journal of the ACM, 2010, 57(2): 7MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Blei D M, Carin L, Dunson D. Probabilistic topic models. IEEE Signal Processing Magazine, 2010, 27(1): 55–65CrossRefGoogle Scholar
  18. 18.
    Blei D M. Probabilistic topic models. Communications of the ACM, 2012, 55(4): 77–84CrossRefGoogle Scholar
  19. 19.
    Xing E P. On topic evolution. Technical Report CMU-CALD-05-115. 2005Google Scholar
  20. 20.
    Teh Y W, Jordan M I, Beal M J, Blei D M. Hierarchical dirichlet processes. Journal of the American Statistical Association, 2006, 101: 1566–1581MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Mei Q Z, Zhai C X. Discovering evolutionary theme patterns from text: an exploration of temporal text mining. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2005, 198–207Google Scholar
  22. 22.
    Nallapati R M, Ditmore S, Lafferty J D, Ung K. Multiscale topic tomography. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2007, 520–529CrossRefGoogle Scholar
  23. 23.
    Ahmed A, Xing E P. Dynamic non-parametric mixture models and the recurrent Chinese restaurant process with application to evolutionary clustering. In: Proceedings of the SIAM International Conference on Data Mining. 2008, 219–230Google Scholar
  24. 24.
    Ahmed A, Xing E P. Timeline: dynamic hierarchical Dirichlet process model for recovering birth/death and evolution of topics in text stream. In: Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence. 2010, 20–29Google Scholar
  25. 25.
    Wang J, Liu X H, Wang J L, Zhao W D. News topic evolution tracking by incorporating temporal information. Communications in Computer and Information Science, 2014, 496(12): 465–472CrossRefGoogle Scholar
  26. 26.
    Wang X R, McCallum A. Topics over time: a non-markov continuoustime model of topical trends. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2006, 424–433CrossRefGoogle Scholar
  27. 27.
    Wang C, Blei D, Heckerman D. Continuous time dynamic topic models. In: Proceedings of the International Conference on Uncertainty in Artificial Intelligence. 2008, 579–586Google Scholar
  28. 28.
    Kawamae N. Trend analysis model: trend consists of temporal words, topics, and timestamps. In: Proceedings of the 4th ACM International Conference on Web Search and Data Mining. 2011, 317–326Google Scholar
  29. 29.
    Dubey A, Hefny A, Williamson S, Xing E P. A nonparametric mixture model for topic modeling over time. In: Proceedings of the SIAM International Conference on Data Mining. 2013, 530–538Google Scholar
  30. 30.
    Li F F, Perona P. A Bayesian hierarchical model for learning natural scene categories. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2005, 524–531Google Scholar
  31. 31.
    Canini K P, Shi L, Griffiths T L. Online inference of topics with latent Dirichlet allocation. In: Proceedings of the International Conference on Artificial Intelligence and Statistics. 2009, 65–72Google Scholar
  32. 32.
    Hoffman M, Bach F R, Blei D M. Online learning for latent dirichlet allocation. In: Proceedings of the Neural Information Processing Systems Conference. 2010, 856–864Google Scholar
  33. 33.
    Sato I, Kurihara K, Nakagawa H. Deterministic single-pass algorithm for LDA. In: Proceedings of the Neural Information Processing Systems Conference. 2010, 2074–2082Google Scholar
  34. 34.
    AlSumait L, Barbará D, Domeniconi C. On-line LDA: adaptive topic models for mining text streams with applications to topic detection and tracking. In: Proceedings of the 8th IEEE International Conference on Data Mining. 2008, 3–12Google Scholar
  35. 35.
    Gohr, A, Hinneburg A, Schult R, Spiliopoulou M. Topic evolution in a stream of documents. In: Proceedings of the SIAM International Conference on Data Mining. 2009, 859–870Google Scholar
  36. 36.
    Iwata T, Yamada T, Sakurai Y, Ueda N. Online multiscale dynamic topic models. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and DataMining. 2010, 663–672Google Scholar
  37. 37.
    Ahmed A, Ho Q, Eisenstein J, Xing E, Smola A J, Teo C H. Unified analysis of streaming news. In: Proceedings of the 20th International Conference on World Wide Web. 2011, 267–276Google Scholar
  38. 38.
    Griffiths T L, Steyvers M. Finding scientific topics. Proceedings of the National Academy of Sciences, 2004, 101 (suppl 1): 5228–5235CrossRefGoogle Scholar
  39. 39.
    Hall D, Jurafsky D, Manning C D. Studying the history of ideas using topic models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2008, 363–371Google Scholar
  40. 40.
    Bolelli L, Ertekin, Giles C L. Topic and trend detection in text collections using latent dirichlet allocation. In: Proceedings of the European Conference on Information Retrieval. 2009, 776–780Google Scholar
  41. 41.
    Steyvers M, Smyth P, Rosen-Zvi M, Griffiths T. Probabilistic authortopic models for information discovery. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004, 306–315Google Scholar
  42. 42.
    Rosen-Zvi M, Griffiths T, Steyvers M, Smyth P. The author-topic model for authors and documents. In: Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. 2004, 487–494Google Scholar
  43. 43.
    Nallapati R M, Ahmed A, Xing E P, Cohen W W. Joint latent topic models for text and citations. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2008, 542–550CrossRefGoogle Scholar
  44. 44.
    Zhou D, Ji X, Zha H Y, Giles C L. Topic evolution and social interactions: how authors effect research. In: Proceedings of the 15th ACM International Conference on Information and Knowledge Management. 2006, 248–257Google Scholar
  45. 45.
    He Q, Chen B, Pei J, Qiu B J, Mitra P, Giles L. Detecting topic evolution in scientific literature: how can citations help? In: Proceedings of the 18th ACM Conference on Information and Knowledge Management. 2009, 957–966Google Scholar
  46. 46.
    Wang X L, Zhai C X, Roth D. Understanding evolution of research themes: a probabilistic generative model for citations. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2013, 1115–1123CrossRefGoogle Scholar
  47. 47.
    Wang X H, Zhai C X, Hu X, Sproat R. Mining correlated bursty topic patterns from coordinated text streams. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2007, 784–793CrossRefGoogle Scholar
  48. 48.
    Hong L J, Dom B, Gurumurthy S, Tsioutsiouliklis K. A timedependent topic model for multiple text streams. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2011, 832–840Google Scholar
  49. 49.
    Lin C X, Zhao B, Mei Q Z, Han J W. PET: a statistical model for popular events tracking in social communities. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2010, 929–938Google Scholar
  50. 50.
    Lin C X, Mei Q Z, Han J W, Jiang Y L, Danilevsky M. The joint inference of topic diffusion and evolution in social communities. In: Proceedings of the 11th IEEE International Conference on Data Mining. 2011, 378–387Google Scholar
  51. 51.
    Tang X N, Yang C C. TUT: a statistical model for detecting trends, topics and user interests in social media. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management. 2012, 972–981Google Scholar
  52. 52.
    Sasaki K, Yoshikawa T, Furuhashi T. Online topic model for twitter considering dynamics of user interests and topic trends. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2014, 1977–1985Google Scholar
  53. 53.
    Iwata T, Watanabe S, Yamada T, Ueda N. Topic tracking model for analyzing consumer purchase behavior. In: Proceedings of the International Joint Conference on Artificial Intelligence. 2009, 1427–1432Google Scholar
  54. 54.
    Cai G Y, Peng L B, Wang Y. Topic detection and evolution analysis on microblog. In: Shi Z Z, Wu Z H, Leake D, et al. eds. Intelligent Information Processing VII. IFIP Adrances in Information and Communication Technology, Vol 432. Berlin: Springer,2014, 67–77Google Scholar
  55. 55.
    Wallach H M, Murray I, Salakhutdinov R, Mimno D. Evaluation methods for topic models. In: Proceedings of the 26th Annual International Conference on Machine Learning. 2009, 1105–1112Google Scholar
  56. 56.
    Saha A, Sindhwani V. Learning evolving and emerging topics in social media: a dynamic nmf approach with temporal regularization. In: Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 2012, 693–702Google Scholar
  57. 57.
    Vaca C K, Mantrach A, Jaimes A, Saerens M. A time-based collective factorization for topic discovery and monitoring in news. In: Proceedings of the 23rd ACM International Conference on World Wide Web. 2014, 527–538Google Scholar
  58. 58.
    Chen Y, Zhang H, Wu J J, Wang X G. Modeling emerging, evolving and fading topics using dynamic soft orthogonal nmf with sparse representation. In: Proceedings of the IEEE International Conference on Data Mining. 2015, 61–70Google Scholar
  59. 59.
    Globerson A, Chechik G, Pereira F, Tishby N. Euclidean embedding of co-occurrence data. The Journal of Machine Learning Research, 2007, 8(4): 2265–2295MathSciNetzbMATHGoogle Scholar
  60. 60.
    Chang J, Boyd-Graber J L, Gerrish S, Wang C, Blei D M. Reading tea leaves: how humans interpret topic models. In: Proceedings of the Neural Information Processing Systems Conference. 2009, 288–296Google Scholar
  61. 61.
    Wallach H M. Topic modeling: beyond bag of words. In: Proceedings of the 23rd International Conference on Machine Learning. 2006, 977–984Google Scholar

Copyright information

© Higher Education Press and Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  1. 1.College of Information Science & Electronic EngineeringZhejiang UniversityHangzhouChina
  2. 2.State Key Laboratory of CAD & CGHangzhouChina
  3. 3.School of Information EngineeringZhejiang A&F UniversityHangzhouChina
  4. 4.Zhejiang Provincial Key Laboratory of Forestry Intelligent Monitoring and Information TechnologyHangzhouChina

Personalised recommendations