Abstract
Topic modelling is a popular unsupervised method for text processing which provides interpretable document representation. One of the most high-level approaches, considering its capability of imitating the behaviour of various methods such as LDA or PLSA, is based on additive regularization technique. However, due to its flexibility and advanced regularization abilities, it is challenging to find optimal learning strategy to create high-quality topics, because a user needs to select the regularizers with their values and determine the order of application. At the same time, there is a lack of research on parameters optimization of topic models, especially for ARTM-based models. Our work proposes an approach that formalizes the learning strategy into a vector of parameters which can be solved with an evolutionary or Bayesian approach. An experimental study conducted on English and Russian datasets indicates that the proposed learning strategy can be successfully optimized even in the presence of strong constrains.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sidorov, G., Herrera-Alcántara, O. (eds.): MICAI 2016. LNCS (LNAI), vol. 10061. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-62434-1
Butakov, N., Petrov, M., Mukhina, K., Nasonov, D., Kovalchuk, S.: Unified domain-specific language for collecting and processing data of social media. J. Int. Inf. Syst. 51, 389–414 (2018)
Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing, 2nd edn. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-44874-8
George, C.P., Doss, H.: Principled selection of hyperparameters in the latent dirichlet allocation model. J. Mach. Learn. Res. 18(1), 5937–5974 (2017)
Hofmann, T.: Probabilistic latent semantic analysis. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, UAI 1999, pp. 289–296. Morgan Kaufmann Publishers Inc., San Francisco (1999)
Lang, K.: NewsWeeder: learning to filter netnews. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 331–339 (1995)
Lau, J.H., Newman, D., Baldwin, T.: Machine reading tea leaves: automatically evaluating topic coherence and topic model quality. In: Proceedings of the 14th Conference of the European Chapter of the ACL, pp. 530–539. ACL, Sweden (2014)
McAuley, J.J., Leskovec, J.: From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. In: Proceedings of the 22nd International Conference on World Wide Web, WWW 2013, pp. 897–908. Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2488388.2488466
Newman, D., Lau, J.H., Grieser, K., Baldwin, T.: Automatic evaluation of topic coherence. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT 2010, pp. 100–108. ACL, USA (2010)
Newman, D., Lau, J.H., Grieser, K., Baldwin, T.: Automatic evaluation of topic coherence. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pp. 100–108. ACL, California (2010)
Nikolenko, S.I.: Topic quality metrics based on distributed word representations. In: Proceedings of the 39th International ACM SIGIR Conference, SIGIR 2016, pp. 1029–1032. Association for Computing Machinery, New York (2016)
Onan, A.: Biomedical text categorization based on ensemble pruning and optimized topic modelling. Comput. Math. Methods Med. 2018, 1–22 (2018)
O’Callaghan, D., Greene, D., Carthy, J., Cunningham, P.: An analysis of the coherence of descriptors in topic modeling. Expert Syst. Appl. 42(13), 5645–5657 (2015). https://doi.org/10.1016/j.eswa.2015.02.055
Panichella, A.: A systematic comparison of search algorithms for topic modelling—a study on duplicate bug report identification. In: Nejati, S., Gay, G. (eds.) SSBSE 2019. LNCS, vol. 11664, pp. 11–26. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27455-9_2
Price, K., Storn, R.M., Lampinen, J.A.: Differential Evolution: A Practical Approach to Global Optimization. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-31306-0
Role, F., Nadif, M.: Handling the impact of low frequency events on co-occurrence based measures of word similarity - a case study of pointwise mutual information (2011)
Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS 2012, pp. 2951–2959. Curran Associates Inc., Red Hook (2012)
Tušar, T., Brockhoff, D., Hansen, N.: Mixed-integer benchmark problems for single- and bi-objective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, pp. 718–726. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3321707.3321868
Vorontsov, K., Frei, O., Apishev, M., Romov, P., Dudarenko, M.: BigARTM: open source library for regularized multimodal topic modeling of large collections. In: Khachay, M.Y., Konstantinova, N., Panchenko, A., Ignatov, D.I., Labunets, V.G. (eds.) AIST 2015. CCIS, vol. 542, pp. 370–381. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26123-2_36
Vorontsov, K., Potapenko, A., Plavin, A.: Additive regularization of topic models for topic selection and sparse factorization. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds.) SLDS 2015. LNCS (LNAI), vol. 9047, pp. 193–202. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17091-6_14
Yutkin, D.: Corpus of Russian news articles collected from lenta.ru. https://github.com/yutkin/Lenta.Ru-News-Dataset
Acknowledgments
This research is financially supported by The Russian Science Foundation, Agreement #20-11-20270.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Khodorchenko, M., Teryoshkin, S., Sokhin, T., Butakov, N. (2020). Optimization of Learning Strategies for ARTM-Based Topic Models. In: de la Cal, E.A., Villar Flecha, J.R., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2020. Lecture Notes in Computer Science(), vol 12344. Springer, Cham. https://doi.org/10.1007/978-3-030-61705-9_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-61705-9_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61704-2
Online ISBN: 978-3-030-61705-9
eBook Packages: Computer ScienceComputer Science (R0)