A Joint Optimization Approach for Personalized Recommendation Diversification

  • Xiaojie Wang
  • Jianzhong Qi
  • Kotagiri Ramamohanarao
  • Yu Sun
  • Bo Li
  • Rui Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10939)


In recommendation systems, items of interest are often classified into categories such as genres of movies. Existing research has shown that diversified recommendations can improve real user experience. However, most existing methods do not consider the fact that users’ levels of interest (i.e., user preferences) in different categories usually vary, and such user preferences are not reflected in the diversified recommendations. We propose an algorithm that considers user preferences for different categories when recommending diversified results, and refer to this problem as personalized recommendation diversification. In the proposed algorithm, a model that captures user preferences for different categories is optimized jointly toward both relevance and diversity. To provide the proposed algorithm with informative training labels and effectively evaluate recommendation diversity, we also propose a new personalized diversity measure. The proposed measure overcomes limitations of existing measures in evaluating recommendation diversity: existing measures either cannot effectively handle user preferences for different categories, or cannot evaluate both relevance and diversity at the same time. Experiments using two real-world datasets confirm the superiority of the proposed algorithm, and show the effectiveness of the proposed measure in capturing user preferences.



This work is supported by Australian Research Council (ARC) Future Fellowships Project FT120100832 and Discovery Project DP180102050.


  1. 1.
    Agrawal, R., Gollapudi, S., Halverson, A., Ieong, S.: Diversifying search results. In: WSDM, pp. 5–14. ACM (2009)Google Scholar
  2. 2.
    Bertin-Mahieux, T., Ellis, D.P., Whitman, B., Lamere, P.: The million song dataset. In: ISMIR, vol. 2, p. 10 (2011)Google Scholar
  3. 3.
    Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR, pp. 335–336 (1998)Google Scholar
  4. 4.
    Clarke, C.L., Kolla, M., Cormack, G.V., Vechtomova, O., Ashkan, A., Büttcher, S., MacKinnon, I.: Novelty and diversity in information retrieval evaluation. In: SIGIR, pp. 659–666. ACM (2008)Google Scholar
  5. 5.
    Dang, V., Croft, W.B.: Diversity by proportionality: an election-based approach to search result diversification. In: SIGIR, pp. 65–74. ACM (2012)Google Scholar
  6. 6.
    Harper, F.M., Konstan, J.A.: The MovieLens datasets: history and context. ACM Trans. Interact. Intell. Syst. (TiiS) 5(4), 19 (2016)Google Scholar
  7. 7.
    Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: ICDM, pp. 263–272. IEEE (2008)Google Scholar
  8. 8.
    Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: SIGKDD, pp. 426–434. ACM (2008)Google Scholar
  9. 9.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS (2013)Google Scholar
  10. 10.
    Radlinski, F., Kleinberg, R., Joachims, T.: Learning diverse rankings with multi-armed bandits. In: ICML, pp. 784–791. ACM (2008)Google Scholar
  11. 11.
    Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: UAI, pp. 452–461 (2009)Google Scholar
  12. 12.
    Sakai, T., Song, R.: Evaluating diversified search results using per-intent graded relevance. In: SIGIR, pp. 1043–1052. ACM (2011)Google Scholar
  13. 13.
    Smyth, B., McClave, P.: Similarity vs. diversity. In: Aha, D.W., Watson, I. (eds.) ICCBR 2001. LNCS (LNAI), vol. 2080, pp. 347–361. Springer, Heidelberg (2001). Scholar
  14. 14.
    Sun, Y., Yuan, N.J., Wang, Y., Xie, X., McDonald, K., Zhang, R.: Contextual intent tracking for personal assistants. In: SIGKDD, pp. 273–282. ACM (2016)Google Scholar
  15. 15.
    Sun, Y., Yuan, N.J., Xie, X., McDonald, K., Zhang, R.: Collaborative nowcasting for contextual recommendation. In: WWW, pp. 1407–1418 (2016)Google Scholar
  16. 16.
    Sun, Y., Yuan, N.J., Xie, X., McDonald, K., Zhang, R.: Collaborative intent prediction with real-time contextual data. TOIS 35(4), 30:1–30:33 (2017)CrossRefGoogle Scholar
  17. 17.
    Tran, T., Phung, D., Venkatesh, S.: Neural choice by elimination via highway networks. In: Cao, H., Li, J., Wang, R. (eds.) PAKDD 2016. LNCS (LNAI), vol. 9794, pp. 15–25. Springer, Cham (2016). Scholar
  18. 18.
    Vargas, S., Baltrunas, L., Karatzoglou, A., Castells, P.: Coverage, redundancy and size-awareness in genre diversity for recommender systems. In: RecSys (2014)Google Scholar
  19. 19.
    Wang, X., Wen, J.R., Dou, Z., Sakai, T., Zhang, R.: Search result diversity evaluation based on intent hierarchies. TKDE 30(1), 156–169 (2018)Google Scholar
  20. 20.
    Wang, Y., Yuan, N.J., Sun, Y., Qin, C., Xie, X.: App download forecasting: an evolutionary hierarchical competition approach. In: IJCAI (2017)Google Scholar
  21. 21.
    Xia, L., Xu, J., Lan, Y., Guo, J., Cheng, X.: Modeling document novelty with neural tensor network for search result diversification. In: SIGIR (2016)Google Scholar
  22. 22.
    Yuan, M., Pavlidis, Y., Jain, M., Caster, K.: Walmart online grocery personalization: behavioral insights and basket recommendations. In: Link, S., Trujillo, J.C. (eds.) ER 2016. LNCS, vol. 9975, pp. 49–64. Springer, Cham (2016). Scholar
  23. 23.
    Ziegler, C.N., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In: WWW, pp. 22–32. ACM (2005)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Xiaojie Wang
    • 1
  • Jianzhong Qi
    • 1
  • Kotagiri Ramamohanarao
    • 1
  • Yu Sun
    • 2
  • Bo Li
    • 3
  • Rui Zhang
    • 1
  1. 1.The University of MelbourneMelbourneAustralia
  2. 2.Twitter Inc.San FranciscoUSA
  3. 3.University of Illinois at Urbana ChampaignChampaignUSA

Personalised recommendations