Modeling Topical Trends over Continuous Time with Priors

  • Tomonari Masada
  • Daiji Fukagawa
  • Atsuhiro Takasu
  • Yuichiro Shibata
  • Kiyoshi Oguri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6064)


In this paper, we propose a new method for topical trend analysis. We model topical trends by per-topic Beta distributions as in Topics over Time (TOT), proposed as an extension of latent Dirichlet allocation (LDA). However, TOT is likely to overfit to timestamp data in extracting latent topics. Therefore, we apply prior distributions to Beta distributions in TOT. Since Beta distribution has no conjugate prior, we devise a trick, where we set one among the two parameters of each per-topic Beta distribution to one based on a Bernoulli trial and apply Gamma distribution as a conjugate prior. Consequently, we can marginalize out the parameters of Beta distributions and thus treat timestamp data in a Bayesian fashion. In the evaluation experiment, we compare our method with LDA and TOT in link detection task on TDT4 dataset. We use word predictive probabilities as term weights and estimate document similarities by using those weights in a TFIDF-like scheme. The results show that our method achieves a moderate fitting to timestamp data.


Beta Distribution Latent Dirichlet Allocation Predictive Probability Word Token Bernoulli Trial 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Topic Detection and Tracking - Phase 4,
  2. 2.
    Allan, J., Lavrenko, V., Nallapati., R.: UMass at TDT 2002. In: Notebook Proceedings of TDT 2002 Workshop (2002)Google Scholar
  3. 3.
    Asuncion, A., Welling, M., Smyth, P., Teh., Y.W.: On Smoothing and Inference for Topic Models. In: UAI 2009 (2009)Google Scholar
  4. 4.
    Blei, D.M., Lafferty, J.D.: Dynamic Topic Models. In: Airoldi, E.M., Blei, D.M., Fienberg, S.E., Goldenberg, A., Xing, E.P., Zheng, A.X. (eds.) ICML 2006. LNCS, vol. 4503, pp. 113–120. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet Allocation. JMLR 3, 993–1022 (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    Cabilio, P., Masaro, J.: Basic Statistical Procedures and Tables. Department of Mathematics and Statistics, Acadia University (2005)Google Scholar
  7. 7.
    Griffiths, T.L., Steyvers, M.: Finding scientific topics. Proc. of Natl. Acad. Sci. 101(suppl.1), 5228–5235 (2004)CrossRefGoogle Scholar
  8. 8.
    Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)zbMATHGoogle Scholar
  9. 9.
    Minka, T.: Estimating a Dirichlet distribution (2000),
  10. 10.
    Nallapati, R.M., Ditmore, S., Lafferty, J.D., Ung, K.: Multiscale Topic Tomography. In: KDD 2007, pp. 520–529 (2007)Google Scholar
  11. 11.
    Ren, L., Dunson, D.B., Carin, L.: The Dynamic Hierarchical Dirichlet Process. In: ICML 2008, pp. 824–831 (2008)Google Scholar
  12. 12.
    Shah, C., Croft, W.B., Jensen, D.: Representing Documents with Named Entities for Story Link Detection. In: CIKM 2006, pp. 868–869 (2006)Google Scholar
  13. 13.
    Masada, T., Fukagawa, D., Takasu, A., Hamada, T., Shibata, Y., Oguri, K.: Dynamic Hyperparameter Optimization for Bayesian Topical Trend Analysis. In: CIKM 2009, pp. 1831–1834 (2009)Google Scholar
  14. 14.
    Wang, C., Blei, D., Heckerman, D.: Continuous Time Dynamic Topic Models. In: UAI 2008, pp. 579–586 (2008)Google Scholar
  15. 15.
    Wang, X.-R., McCallum, A.: Topics over Time: A Non-Markov Continuous-time Model of Topical Trends. In: KDD 2006, pp. 424–433 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Tomonari Masada
    • 1
  • Daiji Fukagawa
    • 2
  • Atsuhiro Takasu
    • 2
  • Yuichiro Shibata
    • 1
  • Kiyoshi Oguri
    • 1
  1. 1.Nagasaki UniversityNagasakiJapan
  2. 2.National Institute of InformaticsTokyoJapan

Personalised recommendations