Abstract
With the explosive growth of user-generated contents (e.g., texts, images and videos) on social networks, it is of great significance to analyze and extract people’s interests from the massive social media data, thus providing more accurate personalized recommendations and services. In this paper, we propose a novel multimodal deep learning algorithm for user profiling, dubbed multi-modal User Attribute Model (mmUAM), which explores the intrinsic semantic correlations across different modalities. Our proposed model is based on Poisson Gamma Belief Network (PGBN), which is a deep learning topic model for count data in documents. By improving PGBN, we succeed in addressing the problem of learning a shared representation between texts and images in order to obtain textual and visual attributes for users. To evaluate the effectiveness of our proposed method, we collect a real dataset from Sina Weibo. Experimental results demonstrate that the proposed algorithm achieves encouraging performance compared with several state-of-the-art methods.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abel, F., Gao, Q., Houben, G.-J., Tao, K.: Semantic enrichment of Twitter posts for user profile construction on the social web. In: Antoniou, G., Grobelnik, M., Simperl, E., Parsia, B., Plexousakis, D., Leenheer, P., Pan, J. (eds.) ESWC 2011. LNCS, vol. 6644, pp. 375–389. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21064-8_26
Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D.M., Jordan, M.I.: Matching words and pictures. JMLR 3, 1107–1135 (2003)
Bian, J., Yang, Y., Chua, T.S.: Multimedia summarization for trending topics in microblogs. In: ACM CIKM, pp. 1807–1812 (2013)
Bian, J., Yang, Y., Chua, T.S.: Predicting trending messages and diffusion participants in microblogging network. In: ACM SIGIR, pp. 537–546 (2014)
Bian, J., Yang, Y., Zhang, H., Chua, T.S.: Multimedia summarization for social events in microblog stream. IEEE Trans. Multimedia 17(2), 216–228 (2015)
Blei, D.M., Jordan, M.I.: Modeling annotated data. In: ACM SIGIR, pp. 127–134 (2003)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. JMLR 3, 993–1022 (2003)
Cheng, X., Yan, X., Lan, Y., Guo, J.: BTM: topic modeling over short texts. IEEE TKDE 26(12), 2928–2941 (2014)
Geng, X., Zhang, H., Song, Z., Yang, Y., Luan, H., Chua, T.S.: One of a kind: User profiling by social curation. In: ACM MM, pp. 567–576 (2014)
He, W., Liu, H., He, J., Tang, S., Du, X.: Extracting interest tags for non-famous users in social network. In: ACM CIKM, pp. 861–870 (2015)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Huang, X., Yang, Y., Hu, Y., Shen, F., Shao, J.: Dynamic user attribute discovery on social media. In: Li, F., Shim, K., Zheng, K., Liu, G. (eds.) APWeb 2016. LNCS, vol. 9931, pp. 256–267. Springer, Cham (2016). doi:10.1007/978-3-319-45814-4_21
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)
Li, X., Cheung, M., She, J.: Connection discovery using shared images by gaussian relational topic model. arxiv:1612.03639 (2016)
Liu, S., Cui, P., Zhu, W., Yang, S.: Learning socially embedded visual representation from scratch. In: ACM MM, pp. 109–118 (2015)
Lowe, D.G.: Object recognition from local scale-invariant features. In: ICCV, vol. 2, pp. 1150–1157 (1999)
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: ICML, pp. 689–696 (2011)
Pang, L., Ngo, C.W.: Mutlimodal learning with deep Boltzmann machine for emotion prediction in user generated videos. In: ICMR, pp. 619–622 (2015)
Rosen-Zvi, M., Griffiths, T., Steyvers, M., Smyth, P.: The author-topic model for authors and documents. In: UAI, pp. 487–494 (2004)
Salakhutdinov, R., Hinton, G.E.: Deep boltzmann machines. In: AISTATS, vol. 1, p. 3 (2009)
Schapire, R.E., Singer, Y.: Boostexter: a boosting-based system for text categorization. Mach. Learn. 39(2–3), 135–168 (2000)
Srivastava, N., Salakhutdinov, R.: Learning representations for multimodal data with deep belief nets. In: ICML Workshop (2012)
Srivastava, N., Salakhutdinov, R.R.: Multimodal learning with deep Boltzmann machines. In: NIPS, pp. 2222–2230 (2012)
Xu, Z., Ru, L., Xiang, L., Yang, Q.: Discovering user interest on twitter with a modified author-topic model. In: IEEE/WIC/ACM WI-IAT, vol. 1, pp. 422–429 (2011)
Yang, X., Li, Y., Luo, J.: Pinterest board recommendation for twitter users. In: ACM MM, pp. 963–966 (2015)
Yang, Y., Zha, Z.J., Gao, Y., Zhu, X., Chua, T.S.: Exploiting web images for semantic video indexing via robust sample-specific loss. IEEE Trans. Multimedia 16(6), 1677–1689 (2014)
Zhao, Y., Liang, S., Ren, Z., Ma, J., Yilmaz, E., de Rijke, M.: Explainable user clustering in short text streams. In: ACM SIGIR, pp. 155–164 (2016)
Zhou, M., Carin, L.: Negative binomial process count and mixture modeling. IEEE T PAMI 37(2), 307–320 (2015)
Zhou, M., Cong, Y., Chen, B.: The poisson gamma belief network. In: NIPS, pp. 3043–3051 (2015)
Zhou, M., Cong, Y., Chen, B.: Augmentable gamma belief networks. JMLR 17(163), 1–44 (2016)
Zhou, M., Hannah, L., Dunson, D.B., Carin, L.: Beta-negative binomial process and poisson factor analysis. In: AISTATS, vol. 22, pp. 1462–1471 (2012)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Project 61572108 and Project 61502081, the National Thousand-Young-Talents Program of China, and the Fundamental Research Funds for the Central Universities under Project ZYGX2014Z007 and Project ZYGX2015J055.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Huang, X., Yang, Z., Yang, Y., Shen, F., Xie, N., Shen, H.T. (2017). A Deep Approach for Multi-modal User Attribute Modeling. In: Huang, Z., Xiao, X., Cao, X. (eds) Databases Theory and Applications. ADC 2017. Lecture Notes in Computer Science(), vol 10538. Springer, Cham. https://doi.org/10.1007/978-3-319-68155-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-68155-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68154-2
Online ISBN: 978-3-319-68155-9
eBook Packages: Computer ScienceComputer Science (R0)