Advertisement

Journal of Intelligent Information Systems

, Volume 52, Issue 1, pp 191–209 | Cite as

Deep recurrent convolutional networks for inferring user interests from social media

  • Jaeyong Kang
  • HongSeok Choi
  • Hyunju LeeEmail author
Article
  • 106 Downloads

Abstract

Online social media services, such as Facebook and Twitter, have recently increased in popularity. Although determining the subjects of individual posts is important for extracting users’ interests from social media, this task is nontrivial because posts are highly contextualized, informal, and limited in length. To address this problem, we propose a deep-neural-network-based approach for predicting user interests in social media. In our framework, a word-embedding technique is used to map the words in social media content into vectors. These vectors are used as input to a bidirectional gated recurrent unit (biGRU). Then, the output of the biGRU and the word-embedding vectors are used to construct a sentence matrix. The sentence matrix is then used as input to a convolutional neural network (CNN) model to predict a user’s interests. Experimental results show that our proposed method combining biGRU and CNN models outperforms existing methods for identifying users’ interests from social media. In addition, posts in social media are sensitive to trends and change with time. Here, we collected posts from two different social media platforms at different time intervals, and trained the proposed model with one set of social media data and tested it with another set of social media data. The experimental results showed that our proposed model can predict users’ interests from the independent data set with high accuracies.

Keywords

Text mining User profile Deep learning Text categorization Recommendation systems Social media 

Notes

Acknowledgements

This research was supported by the Bio-Synergy Research Project (NRF-2016M3A9C4939665) of the Ministry of Science, ICT and Future Planning through the National Research Foundation, and by Ministry of Culture, Sports and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program 2016.

References

  1. Bergstra, J., Breuleux, O., Bastien, F.F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio Y. (2010). Theano: a cpu and gpu math compiler in python. In Proceedings of the Python for Scientific Computing Conference (SciPy) (pp. 1–7).Google Scholar
  2. Bhattacharya, P., Zafar, M., Ganguly, N., Ghosh, S., Gummadi, K. (2014). Inferring user interests in the twitter social network. In Proceedings of the 8th ACM Conference on Recommender systems.Google Scholar
  3. Blei, D., Ng, A., Jordan, M. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.zbMATHGoogle Scholar
  4. Budak, C., Kannan, A., Agrawal, R., Pedersen, J. (2014). Inferring user interests from microblogs. Technical report, Microsoft research.Google Scholar
  5. Chen, J., Nairn, R., Nelson, L., Bernstein, M. (2010). Short and tweet: experiments on recommending content from information streams. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1185–1194).Google Scholar
  6. Chetlur, S., & Woolley, C. (2014). cudnn: Efficient primitives for deep learning. arXiv pp. 1–9.Google Scholar
  7. Cho, K., Merrienboer, B.V., Bahdanau, D., Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv.Google Scholar
  8. Chung, J., Gulcehre, C., Cho, K., Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.Google Scholar
  9. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P. (2011). Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12, 2493–2537.zbMATHGoogle Scholar
  10. Firth, J. (1957). A synopsis of linguistic theory. Studies in linguistic analysis.Google Scholar
  11. Gao, J., He, X., Yih, W., Deng, L. (2014). Learning continuous phrase representations for translation modeling. In Proceedings of the 52nd annual meeting of the association for computational linguistics (pp. 699–709).Google Scholar
  12. Godoy, D., & Amandi, A. (2006). Modeling user interests by conceptual clustering. Information Systems, 31, 247–265.CrossRefzbMATHGoogle Scholar
  13. Goldberg, Y., & Levy, O. (2014). word2vec explained: deriving mikolov et al.’s negative-sampling word-embedding method. CoRR arXiv:1402.3722.
  14. Gorrell, G. (2006). Generalized hebbian algorithm for incremental singular value decomposition in natural language processing. In 11th conference of the European chapter of the association for computational linguistics.Google Scholar
  15. Han, J., & Lee, H. (2014). Characterizing user interest using heterogeneous media. In Proceedings of the 23nd international conference on World Wide Web (WWW Companion’14) (pp. 289–290).Google Scholar
  16. Han, J., & Lee, H. (2016). Characterizing the interests of social media users: Refinement of a topic model for incorporating heterogeneous media. Information Sciences, 358, 112–128.CrossRefGoogle Scholar
  17. Han, J., Xie, X., Woo, W. (2013). Context-based microblog browsing for mobile users. Journal of Ambient Intelligence and Smart Environments, 5, 89–104.Google Scholar
  18. Harris, Z.S. (1954). Distributional structure. Word, 10, 146–162.CrossRefGoogle Scholar
  19. Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv:12070580.
  20. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, pp. 1735–1780.Google Scholar
  21. Johnson, R., & Zhang, T. (2015). Semi-supervised convolutional neural networks for text categorization via region embedding. In Advances in Neural Information Processing Systems (NIPS) (pp. 919–927).Google Scholar
  22. Kalchbrenner, N., Grefenstette, E., Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd annual meeting of the association for computational linguistics (pp. 655–665).Google Scholar
  23. Kang, J., & Lee, H. (2017). Modeling user interest in social media using news media and wikipedia. Information Systems, 65, 52–64.CrossRefGoogle Scholar
  24. Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the conference on empirical methods in natural language processing (EMNLP ’14) (pp. 1746–1751).Google Scholar
  25. Long, J., Shelhamer, E., Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440).Google Scholar
  26. Mikolov, T., Chen, K., Corrado, G., Dean, J. (2013a). Efficient estimation of word representations in vector space. CoRR arXiv:1301.3781.
  27. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111–3119).Google Scholar
  28. Nair, V., & Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML ’10).Google Scholar
  29. Ottoni, R., Casas, D.L., Pesce, J., Meira, W. Jr, Wilson, C., Mislove, A., Almeida, V. (2014). Of pins and tweets: Investigating how users behave across image-and text-based social networks. In Proceedings of the International Conference on Web and Social Media (ICWSM ’14).Google Scholar
  30. Pennington, J., Socher, R., Manning, C.D. (2014). Glove: Global vectors for word representation. In Proceedings of the conference on empirical methods in natural language processing.Google Scholar
  31. Ramanathan, K., & Kapoor, K. (2009). Creating user profiles using Wikipedia, vol. 5829, (pp. 415–427). Berlin: Springer.Google Scholar
  32. Rong, X. (2014). word2vec parameter learning explained. CoRR arXiv:1411.2738.
  33. Rothe, S., Ebert, S., Schutze, H. (2016). Ultradense word embeddings by orthogonal transformation. arXiv.Google Scholar
  34. Rummelhart, D.E., McClelland, J.L., Group, P.R., et al. (1986). Parallel distributed processing, vol 1. Explorations in the microstructure of cognition.Google Scholar
  35. Salton, G., & McGill, M. (1986). Introduction to modern information retrieval. New York: McGraw-Hill Inc.zbMATHGoogle Scholar
  36. Schutze, H. (1992). Dimensions of meaning. In Proceedings of IEEE Supercomputing (pp. 787–796).Google Scholar
  37. Severyn, A., & Moschitti, A. (2015a). Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference (pp. 959–962).Google Scholar
  38. Severyn, A., & Moschitti, A. (2015b). Unitn: Training deep convolutional neural network for twitter sentiment classification. In Proceedings of the 9th international workshop on semantic evaluation.Google Scholar
  39. Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G. (2014a). A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management (pp. 101–110).Google Scholar
  40. Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G. (2014b). Learning semantic representations using convolutional neural networks for web search. In Proceedings of the companion publication of the 23rd international conference on World wide web companion (pp. 373–374).Google Scholar
  41. Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP ’13) (pp. 1631–1642).Google Scholar
  42. Sriram, B., Fuhry, D., Demir, E., Ferhatosmanoglu, H., Demirbas, M. (2010). Short text classification in twitter to improve information filtering. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval (pp. 841–842).Google Scholar
  43. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9).Google Scholar
  44. Weng, J., Lim, E., Jiang, J., He, Q. (2010). Twitterrank: finding topic-sensitive influential twitterers. In Proceedings of the third ACM international conference on Web search and data mining (WSDM ’10) (pp. 261–270).Google Scholar
  45. Yang, S., Kolcz, A., Schlaikjer, A., Gupta, P. (2014). Large-scale high-precision topic modeling on twitter. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining.Google Scholar
  46. Yih, W., He, X., Meek, C. (2014). Semantic parsing for single-relation question answering. In Proceedings of ACL (pp. 643–648).Google Scholar
  47. Zeiler, M.D. (2012). Adadelta: an adaptive learning rate method. CoRR arXiv:1212.5701.
  48. Zhang, X., Zhao, J., LeCun, Y. (2015). Character-level convolutional networks for text classification. In Proceedings of the advances in neural information processing systems (pp. 649–657).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018
corrected publication November/2018

Authors and Affiliations

  1. 1.School of Electrical Engineering and Computer ScienceGwangju Institute of Science and TechnologyGwangjuSouth Korea

Personalised recommendations