A Full-Cycle Methodology for News Topic Modeling and User Feedback Research

  • Sergei Koltsov
  • Sergei Pashakhin
  • Sofia Dokuka
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11185)


Online social networks (OSNs) play an increasingly important role in news dissemination and consumption, attracting such traditional media outlets as TV channels with growing online audiences. Online news streams require appropriate instruments for analysis. One of such tools is topic modeling (TM). However, TM has a set of limitations (the problem of topic number choice and the algorithm instability, among others) that must be addressed specifically for the task of sociological online news analysis. In this paper, we propose a full-cycle methodology for such study: from choosing the optimal topic number to the extraction of stable topics and analysis of TM results. We illustrate it with an analysis of online news stream of 164,426 messages formed by twelve national TV channels during a one-year period in a leading Russian OSN. We show that our method can easily reveal associations between news topics and user feedback, including sharing behavior. Additionally, we show how uneven distribution of document quantities and lengths over classes (TV channels) could affect TM results.


Topic modeling Text mining TV news News consumptions Online social networks Social media 



The reported study was funded by RFBR according to the research project № 18-011-00997 A.


  1. 1.
    Daud, A., Li, J., Zhou, L., Muhammad, F.: Knowledge discovery through directed probabilistic topic models: a survey. Front. Comput. Sci. China 4, 280–301 (2010). Scholar
  2. 2.
    Ziegele, M., Breiner, T., Quiring, O.: An exploratory analysis of discussion factors in user comments on news items: what creates interactivity in online news discussions? J. Commun. 64, 1111–1138 (2014). Scholar
  3. 3.
    Bobkowski, P.S.: Sharing the news: effects of informational utility and opinion leadership on online news sharing. J. Mass Commun. Q. 92, 320–345 (2015). Scholar
  4. 4.
    Landau, D.P., Binder, R.: A guide to Monte Carlo simulations in statistical physics. Cambridge University Press, Cambridge (2009)Google Scholar
  5. 5.
    Griffiths, T.L., Steyvers, M.: Finding scientific topics. PNAS 101, 5228–5235 (2004). Scholar
  6. 6.
    Cao, J., Xia, T., Li, J., Zhang, Y., Tang, S.: A density-based method for adaptive LDA model selection. Neurocomputing 72, 1775–1781 (2009). Scholar
  7. 7.
    Arun, R., Suresh, V., Veni Madhavan, C.E., Narasimha Murthy, M.N.: On finding the natural number of topics with latent Dirichlet allocation: some observations. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (eds.) PAKDD 2010. LNCS (LNAI), vol. 6118, pp. 391–402. Springer, Heidelberg (2010). Scholar
  8. 8.
    Roberts, M.E., Stewart, B.M., Tingley, D.: Navigating the local modes of big data: the case of topic models. In: Alvarez, R.M. (ed.) Computational Social Science, pp. 51–97. Cambridge University Press, Cambridge (2016)CrossRefGoogle Scholar
  9. 9.
    Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical Dirichlet processes. J. Am. Stat. Assoc. 101, 1566–1581 (2006)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Blei, D.M., Griffiths, T.L., Jordan, M.I., Tenebaum, J.B.: Hierarchical topic models and the nested Chinese restaurant process. Adv. Neural. Inf. Process. Syst. 16, 17–24 (2004). Scholar
  11. 11.
    Koltcov, S.N.: A thermodynamic approach to selecting a number of clusters based on topic modeling. Tech. Phys. Lett. 43, 584–586 (2017). Scholar
  12. 12.
    Beck, C., Schlögl, F.: Thermodynamics of Chaotic Systems. Cambridge University Press, Cambridge (1993)Google Scholar
  13. 13.
    Koltcov, S., Nikolenko, Sergey I., Koltsova, O., Filippov, V., Bodrunova, S.: Stable topic modeling with local density regularization. In: Bagnoli, F., Satsiou, A., Stavrakakis, I., Nesi, P., Pacini, G., Welp, Y., Tiropanis, T., DiFranzo, D. (eds.) INSCI 2016. LNCS, vol. 9934, pp. 176–188. Springer, Cham (2016). Scholar
  14. 14.
    Koltcov, S., Koltsova, O., Nikolenko, S.: Latent Dirichlet allocation: stability and applications to studies of user-generated content. In: Proceedings of the 2014 ACM Conference on Web Science. ACM, Bloomington, Indiana, USA, pp. 161–165 (2014)Google Scholar
  15. 15.
    Blei, D.M., Lafferty, J.D.: Topic models. In: Text Mining: Classification, Clustering, and Applications, pp. 71–94. CRC Press (2009)Google Scholar
  16. 16.
    Nagornyy, O., Koltsova, O.: Mining media topics perceived as social problems by online audiences: use of a data mining approach in sociology. Higher School of Economics Research Paper No. WP BRP 74/SOC/2017. SSRN (2017). or
  17. 17.
    Prozorov, S.: Russian conservatism in the Putin presidency: the dispersion of a hegemonic discourse. J. Polit. Ideol. 10, 121–143 (2005). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.National Research University Higher School of EconomicsSt. PetersburgRussia
  2. 2.Institute of EducationNational Research University Higher School of EconomicsMoscowRussia

Personalised recommendations