Deep recurrent convolutional networks for inferring user interests from social media
- 106 Downloads
Online social media services, such as Facebook and Twitter, have recently increased in popularity. Although determining the subjects of individual posts is important for extracting users’ interests from social media, this task is nontrivial because posts are highly contextualized, informal, and limited in length. To address this problem, we propose a deep-neural-network-based approach for predicting user interests in social media. In our framework, a word-embedding technique is used to map the words in social media content into vectors. These vectors are used as input to a bidirectional gated recurrent unit (biGRU). Then, the output of the biGRU and the word-embedding vectors are used to construct a sentence matrix. The sentence matrix is then used as input to a convolutional neural network (CNN) model to predict a user’s interests. Experimental results show that our proposed method combining biGRU and CNN models outperforms existing methods for identifying users’ interests from social media. In addition, posts in social media are sensitive to trends and change with time. Here, we collected posts from two different social media platforms at different time intervals, and trained the proposed model with one set of social media data and tested it with another set of social media data. The experimental results showed that our proposed model can predict users’ interests from the independent data set with high accuracies.
KeywordsText mining User profile Deep learning Text categorization Recommendation systems Social media
This research was supported by the Bio-Synergy Research Project (NRF-2016M3A9C4939665) of the Ministry of Science, ICT and Future Planning through the National Research Foundation, and by Ministry of Culture, Sports and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program 2016.
- Bergstra, J., Breuleux, O., Bastien, F.F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., Bengio Y. (2010). Theano: a cpu and gpu math compiler in python. In Proceedings of the Python for Scientific Computing Conference (SciPy) (pp. 1–7).Google Scholar
- Bhattacharya, P., Zafar, M., Ganguly, N., Ghosh, S., Gummadi, K. (2014). Inferring user interests in the twitter social network. In Proceedings of the 8th ACM Conference on Recommender systems.Google Scholar
- Budak, C., Kannan, A., Agrawal, R., Pedersen, J. (2014). Inferring user interests from microblogs. Technical report, Microsoft research.Google Scholar
- Chen, J., Nairn, R., Nelson, L., Bernstein, M. (2010). Short and tweet: experiments on recommending content from information streams. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1185–1194).Google Scholar
- Chetlur, S., & Woolley, C. (2014). cudnn: Efficient primitives for deep learning. arXiv pp. 1–9.Google Scholar
- Cho, K., Merrienboer, B.V., Bahdanau, D., Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv.Google Scholar
- Chung, J., Gulcehre, C., Cho, K., Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.Google Scholar
- Firth, J. (1957). A synopsis of linguistic theory. Studies in linguistic analysis.Google Scholar
- Gao, J., He, X., Yih, W., Deng, L. (2014). Learning continuous phrase representations for translation modeling. In Proceedings of the 52nd annual meeting of the association for computational linguistics (pp. 699–709).Google Scholar
- Goldberg, Y., & Levy, O. (2014). word2vec explained: deriving mikolov et al.’s negative-sampling word-embedding method. CoRR arXiv:1402.3722.
- Gorrell, G. (2006). Generalized hebbian algorithm for incremental singular value decomposition in natural language processing. In 11th conference of the European chapter of the association for computational linguistics.Google Scholar
- Han, J., & Lee, H. (2014). Characterizing user interest using heterogeneous media. In Proceedings of the 23nd international conference on World Wide Web (WWW Companion’14) (pp. 289–290).Google Scholar
- Han, J., Xie, X., Woo, W. (2013). Context-based microblog browsing for mobile users. Journal of Ambient Intelligence and Smart Environments, 5, 89–104.Google Scholar
- Hinton, G., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv:12070580.
- Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, pp. 1735–1780.Google Scholar
- Johnson, R., & Zhang, T. (2015). Semi-supervised convolutional neural networks for text categorization via region embedding. In Advances in Neural Information Processing Systems (NIPS) (pp. 919–927).Google Scholar
- Kalchbrenner, N., Grefenstette, E., Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd annual meeting of the association for computational linguistics (pp. 655–665).Google Scholar
- Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the conference on empirical methods in natural language processing (EMNLP ’14) (pp. 1746–1751).Google Scholar
- Long, J., Shelhamer, E., Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431–3440).Google Scholar
- Mikolov, T., Chen, K., Corrado, G., Dean, J. (2013a). Efficient estimation of word representations in vector space. CoRR arXiv:1301.3781.
- Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J. (2013b). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111–3119).Google Scholar
- Nair, V., & Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML ’10).Google Scholar
- Ottoni, R., Casas, D.L., Pesce, J., Meira, W. Jr, Wilson, C., Mislove, A., Almeida, V. (2014). Of pins and tweets: Investigating how users behave across image-and text-based social networks. In Proceedings of the International Conference on Web and Social Media (ICWSM ’14).Google Scholar
- Pennington, J., Socher, R., Manning, C.D. (2014). Glove: Global vectors for word representation. In Proceedings of the conference on empirical methods in natural language processing.Google Scholar
- Ramanathan, K., & Kapoor, K. (2009). Creating user profiles using Wikipedia, vol. 5829, (pp. 415–427). Berlin: Springer.Google Scholar
- Rong, X. (2014). word2vec parameter learning explained. CoRR arXiv:1411.2738.
- Rothe, S., Ebert, S., Schutze, H. (2016). Ultradense word embeddings by orthogonal transformation. arXiv.Google Scholar
- Rummelhart, D.E., McClelland, J.L., Group, P.R., et al. (1986). Parallel distributed processing, vol 1. Explorations in the microstructure of cognition.Google Scholar
- Schutze, H. (1992). Dimensions of meaning. In Proceedings of IEEE Supercomputing (pp. 787–796).Google Scholar
- Severyn, A., & Moschitti, A. (2015a). Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference (pp. 959–962).Google Scholar
- Severyn, A., & Moschitti, A. (2015b). Unitn: Training deep convolutional neural network for twitter sentiment classification. In Proceedings of the 9th international workshop on semantic evaluation.Google Scholar
- Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G. (2014a). A latent semantic model with convolutional-pooling structure for information retrieval. In Proceedings of the 23rd ACM international conference on conference on information and knowledge management (pp. 101–110).Google Scholar
- Shen, Y., He, X., Gao, J., Deng, L., Mesnil, G. (2014b). Learning semantic representations using convolutional neural networks for web search. In Proceedings of the companion publication of the 23rd international conference on World wide web companion (pp. 373–374).Google Scholar
- Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP ’13) (pp. 1631–1642).Google Scholar
- Sriram, B., Fuhry, D., Demir, E., Ferhatosmanoglu, H., Demirbas, M. (2010). Short text classification in twitter to improve information filtering. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval (pp. 841–842).Google Scholar
- Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9).Google Scholar
- Weng, J., Lim, E., Jiang, J., He, Q. (2010). Twitterrank: finding topic-sensitive influential twitterers. In Proceedings of the third ACM international conference on Web search and data mining (WSDM ’10) (pp. 261–270).Google Scholar
- Yang, S., Kolcz, A., Schlaikjer, A., Gupta, P. (2014). Large-scale high-precision topic modeling on twitter. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining.Google Scholar
- Yih, W., He, X., Meek, C. (2014). Semantic parsing for single-relation question answering. In Proceedings of ACL (pp. 643–648).Google Scholar
- Zeiler, M.D. (2012). Adadelta: an adaptive learning rate method. CoRR arXiv:1212.5701.
- Zhang, X., Zhao, J., LeCun, Y. (2015). Character-level convolutional networks for text classification. In Proceedings of the advances in neural information processing systems (pp. 649–657).Google Scholar