Abstract
Modeling point-of-Interest (POI) for recommendations is vital in location-based social networks, yet it is a challenging task due to data sparsity and cold-start problems. Most existing approaches incorporate content features into a probabilistic matrix factorization model using unsupervised learning, which results in inaccuracy and weak robustness of recommendations when data is sparse, and the cold-start problems remain unsolved. In this paper, we propose a deep multimodal rank learning (DMRL) model that improves both the accuracy and robustness of POI recommendations. DMRL exploits temporal dynamics by allowing each user to have time-dependent preferences and captures geographical influences by introducing spatial regularization to the model. DMRL jointly learns ranking for personal preferences and supervised deep learning models to create a semantic representation of POIs from multimodal content. To make model optimization converge more rapidly while preserving high effectiveness, we develop a ranking-based dynamic sampling strategy to sample adverse or negative POIs for model training. We conduct experiments to compare our DMRL model with existing models that use different approaches using two large-scale datasets obtained from Foursquare and Yelp. The experimental results demonstrate the superiority of DMRL over the other models in creating cold-start POI recommendations and achieving excellent and highly robust results for different degrees of data sparsity.
Similar content being viewed by others
References
Guo, L., Yin, H., Wang, Q., Chen, T., Zhou, A., Hung, N.Q.V.: Streaming Session-based Recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Pp.1569–1577. ACM Press, Anchorage, AK, USA (2019). https://doi.org/10.1145/3292500.3330839
Chen, T., Yin, H., Chen, H., Yan, R., Nguyen, Q.V.H., Li, X.: AIR: Attentional Intention-Aware Recommender Systems. In: Proceedings of the 35th International Conference on Data Engineering. pp. 304–315. IEEE Press, Macao, China(2019). https://doi.org/10.1109/ICDE.2019.00035
Wang, S., Hu, L., Wang, Y., Cao, L., Sheng, Q.Z., Orgun, M.: Sequential Recommender Systems: Challenges, Progress and Prospects. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 6332–6338. AAAI Press, Macao (2019)
Liu, Y., Pham, T.-A.N., Cong, G., Yuan, Q.: An experimental evaluation of point-of-interest recommendation in location-based social networks. Proc. VLDB Endow. 10, 1010–1021 (2017). https://doi.org/10.14778/3115404.3115407
Li, J., Lu, K., Huang, Z., Shen, H.T.: Two Birds One Stone: On both Cold-Start and Long-Tail Recommendation. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 898–906. ACM Press, California (2017)
Gao, H., Tang, J., Hu, X., Liu, H.: Content-aware point of interest recommendation on location-based social networks. In: Proceedings of the 29th AAAI conference on artificial intelligence. pp. 1721–1727.AAAI Press, Austin, Texas (2015)
Guo, H., Li, H., He, M., Zhao,X.Y., Liu, G.Q., Xu GD.: Joint Factor Model with Content, Social, Location for Heterogeneous Point-of-Interest Recommendation. In: Proceedings of the International Conference on Knowledge Science, Engineering and Management. pp. 613–627. Springer Press, Passau, Germany (2016)
Yin, H., Zhou, X., Cui, B., Wang, H., Zheng, K., Nguyen, Q.V.H.: Adapting to user interest drift for POI recommendation. IEEE Trans. Knowl. Data Eng. 28, 2566–2581 (2016). https://doi.org/10.1109/TKDE.2016.2580511
Wang, H., Wang, N., Yeung, D.-Y.: Collaborative deep learning for recommender systems. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1235–1244. ACM Press, Sydney (2015)
Kim, D., Park, C., Oh, J., Lee, S., Yu, H.: Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM conference on recommender systems, pp. 233–240. ACM Press, Boston (2016)
Zhang, Y., Yin, H., Huang, Z., Du, X., Yang, G., Lian, D.: Discrete deep learning for fast content-aware recommendation. In: Proceedings of the eleventh ACM international conference on web search and data mining, pp. 717–726. ACM Press, Marina Del Rey (2018)
Yin, H., Wang, W., Wang, H., Chen, L., Zhou, X.: Spatial-aware hierarchical collaborative deep learning for POI recommendation. IEEE Trans. Knowl. Data Eng. 29, 2537–2551 (2017). https://doi.org/10.1109/TKDE.2017.2741484
Liu, Q., Wu, S., Wang, L., Tan, T.: Predicting the Next Location: A Recurrent Model with Spatial and Temporal Contexts. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence, pp. 194–200. AAAI Press, Phoenix (2016)
Liao, J., Liu, T., Liu, M., Wang, J., Wang, Y., Sun, H.: Multi-context integrated deep neural network model for next location prediction. IEEE Access. 6, 21980–21990 (2018). https://doi.org/10.1109/ACCESS.2018.2827422
Cheng, C., Yang, H., King, I., Lyu, M.R.: Fused Matrix Factorization with Geographical and Social Influence in Location-Based Social Networks. In: Proceedings of the 26th AAAI Conference on Artificial Intelligence, pp. 17–23. AAAI Press, Toronto (2012)
Lian, D., Zhao, C., Xie, X., Sun, G., Chen, E., Rui, Y., GeoMF: Joint geographical modeling and matrix factorization for point-of-interest recommendation. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 831–840. ACM Press, New York (2014)
Liu, Y., Wei, W., Sun, A., Miao, C.: Exploiting Geographical Neighborhood Characteristics for Location Recommendation. In: Proceedings of the 23rd ACM International Conference on Information and Knowledge Management, pp. 739–748. ACM Press, Shanghai (2014)
Wang, H., Shen, H., Ouyang, W., Cheng, X.: Exploiting POI-Specific Geographical Influence for Point-of-Interest Recommendation. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. pp. 3877–3883. IJCAI Press, Stockholm (2018)
Liu, B., Xiong, H., Papadimitriou, S., Fu, Y., Yao, Z.: A general geographical probabilistic factor model for point of interest recommendation. IEEE Trans. Knowl. Data Eng. 27, 1167–1179 (2015). https://doi.org/10.1109/TKDE.2014.2362525
Li, X., Jiang, M., Hong, H., Liao, L.: A time-aware personalized point-of-interest recommendation via high-order tensor factorization. ACM Trans. Inf. Syst. 35, 1–23 (2017). https://doi.org/10.1145/3057283
Yao, L., Sheng, Q.Z., Wang, X., Zhang, W.E., Qin, Y.: Collaborative location recommendation by integrating multi-dimensional contextual information. ACM Trans. Internet Technol. 18, 1–24 (2018). https://doi.org/10.1145/3134438
Liu, B., Xiong, H.: Point-of-interest recommendation in location based social networks with topic and location awareness. In: Proceedings of the 2013 SIAM international conference on data mining, pp. 396–404. SIAM, Philadelphia (2013)
Li, X., Xu, G.D., Chen, E.H., Li, L.: MARS: A multi-aspect Recommender system for Point-of-Interest. In: Proceedings of the IEEE 31st International Conference on Data Engineering, pp. 1436–1439. IEEE press, South Korea (2015)
Wang, H., Fu, Y., Wang, Q., Yin, H., Du, C., Xiong, H.: A location-sentiment-aware recommender system for both home-town and out-of-town users. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135–1143. ACM Press, Halifax (2017)
Xie, M., Yin, H., Wang, H., Xu, F., Chen, W., Wang, S.: Learning Graph-based POI Embedding for Location-based Recommendation. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 15–24. ACM Press, Indianapolis (2016)
Zhang, F., Yuan, N.J., Lian, D., Xie, X., Ma, W.-Y.: Collaborative Knowledge Base embedding for recommender systems. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 353–362. ACM Press, San Francisco (2016)
Wang, S., Wang, Y., Tang, J., Shu, K., Ranganath, S., Liu, H.: What your images reveal: exploiting visual contents for point-of-interest recommendation. In: Proceedings of the 26th international conference on world wide web, pp. 391–400. ACM Press, Perth (2017)
Li, P., Burges, C.J.C., Wu, Q.: Learning to Rank Using Classification and Gradient Boosting. In: Advances in Neural Information Processing Systems 20. p. 10 (2008)
Jing He, L.L., Xin Li: Category-aware Next Point-of-Interest Recommendation via Listwise Bayesian Personalized Ranking. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. pp. 1837–1843. IJCAI Press, Melbourne (2017)
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence, pp. 452–461. AUAI Press, Montreal (2009)
Rendle, S., Freudenthaler, C.: Improving pairwise learning for item recommendation from implicit feedback. In: Proceedings of the 7th ACM international conference on web search and data mining, pp. 273–282. ACM Press, New York (2014)
Feng, S., Li, X., Zeng, Y., Cong, G., Chee, Y.M., Yuan, Q.: Personalized Ranking Metric Embedding for Next New POI Recommendation. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence, pp. 2069–2075. AAAI Press, New York (2015)
Li, X., Cong, G., Li, X.-L., Pham, T.-A.N., Krishnaswamy, S.: Rank-GeoFM: A Ranking based Geographical Factorization Method for Point of Interest Recommendation. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 433–442. ACM Press, Santiago (2015)
Zhao, S., Zhao, T., Yang, H., Lyu, M.R., King, I.: STELLAR: Spatial-Temporal Latent Ranking for Successive Point-of-Interest Recommendation. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence, pp. 315–322. AAAI Press, Phoenix (2016)
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal Deep Learning. In: Proceedings of the 28th International Conference on Machine Learning. pp. 689–696. ACM Press, Bellevue (2011)
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., Cichocki, A.: EmotionMeter: A Multimodal Framework for Recognizing Human Emotions. IEEE Trans. Cybern. 1–13 (2018). doi:https://doi.org/10.1109/TCYB.2018.2797176
Wang, B., Yang, Y., Xu, X., Hanjalic, A., Shen, H.T.: Adversarial Cross-Modal Retrieval. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 154–162. ACM Press, Mountain View (2017)
Oramas, S., Nieto, O., Sordo, M., Serra, X.: A Deep Multimodal Approach for Cold-start Music Recommendation. ArXiv170609739 Cs. 32–37 (2017). doi:https://doi.org/10.1145/3125486.3125492
Zhao, Z., Yang, Q., Lu, H., Weninger, T., Cai, D., He, X., Zhuang, Y.: Social-aware movie recommendation via multimodal network learning. IEEE Trans. Multimed. 20, 430–440 (2018). https://doi.org/10.1109/TMM.2017.2740022
Wang, X., Zhao, Y.-L., Nie, L., Gao, Y., Nie, W., Zha, Z.-J., Chua, T.-S.: Semantic-based location recommendation with multimodal venue semantics. IEEE Trans. Multimed. 17, 409–419 (2015). https://doi.org/10.1109/TMM.2014.2385473
He, R., McAuley, J.: VBPR: visual Bayesian personalized ranking from implicit feedback. In: Proceedings of the 30th AAAI conference on artificial intelligence. pp. 144–150. AAAI Press, Phoenix (2016)
Srivastava, N., Mansimov, E., Salakhutdinov, R.: Unsupervised learning of video representations using LSTMs. In: Proceedings of the 32nd international conference on machine learning. pp. 843–852. ACM Press, Lille (2015)
Yin, H., Chen, H., Sun, X., Wang, H., Wang, Y., Nguyen, Q.V.H.: SPTF: A Scalable Probabilistic Tensor Factorization Model for Semantic-Aware Behavior Prediction. In: 2017 IEEE International Conference on Data Mining, pp. 585–594. IEEE, New Orleans (2017)
Liu, Y., Yang, J.: Improving ranking-based recommendation by social information and negative similarity. Procedia Comput. Sci. 55, 732–740 (2015). https://doi.org/10.1016/j.procs.2015.07.164
Yang, D., Zhang, D., Zheng, V.W., Yu, Z.: Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs. IEEE Trans. Syst. Man Cybern. Syst. 45, 129–142 (2015). https://doi.org/10.1109/TSMC.2014.2327053
Yin, H., Zou, L., Nguyen, Q.V.H., Huang, Z., Zhou, X.: Joint Event-Partner Recommendation in Event-based Social Networks. In: Proceedings of the 34th IEEE International Conference on Data Engineering. pp.929-940. IEEE Press, Paris, France (2018)
Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 448–456. ACM Press, San Diego (2011)
Mnih, A., Salakhutdinov, R.R.: Probabilistic matrix factorization. In: Proceedings of the 20th international conference on neural information processing systems. pp. 1257–1264. MIT Press, Vancouver (2007)
Acknowledgments
This work was supported in part by the National Key R&D Program of China 2020YFB1807800???2020YFB1807804, in part by the National Natural Science Foundation of China under Grants 62071067,62001054 and 61771068, in part by the Beijing Municipal Natural Science Foundation under Grant 4182041, and in part by the National Postdoctoral Program for Innovative Talents under Grant BX20200067, and in part by the Ministry of Education and China Mobile Joint Fund MCM20180101, and in part by the Beijing University of Posts and Telecommunications-China Mobile Research Institute Joint Innovation Center.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liao, J., Liu, T., Yin, H. et al. An integrated model based on deep multimodal and rank learning for point-of-interest recommendation. World Wide Web 24, 631–655 (2021). https://doi.org/10.1007/s11280-021-00865-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11280-021-00865-8