Skip to main content
SpringerLink
Log in

Integrating optimized item selection with active learning for continuous exploration in recommender systems

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

Recommender Systems have become the backbone of personalized services that provide tailored experiences to individual users, yet designing new recommendation applications with limited or no available training data remains a challenge. To address this issue, we focus on selecting the universe of items for experimentation in recommender systems by leveraging a recently introduced combinatorial problem. On the one hand, selecting a large set of items is desirable to increase the diversity of items. On the other hand, a smaller set of items enables rapid experimentation and minimizes the time and the amount of data required to train machine learning models. We first present how to optimize for such conflicting criteria using a multi-level optimization framework. Then, we shift our focus to the operational setting of a recommender system. In practice, to work effectively in a dynamic environment where new items are introduced to the system, we need to explore users’ behaviors and interests continuously. To that end, we show how to integrate the item selection approach with active learning to guide randomized exploration in an ongoing fashion. Our hybrid approach combines techniques from discrete optimization, unsupervised clustering, and latent text embeddings. Experimental results on well-known movie and book recommendation benchmarks demonstrate the benefits of optimized item selection and efficient exploration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Availability of data and materials

The dataset used in our experiments are publicly available benchmarks that are commonly used in this line of research. 1. The Goodreads datasets [32, 33] are available at https://sites.google.com/eng.ucsd.edu/ucsdbookgraph/home. 2. The MovieLens datasets [34] are available at https://grouplens.org/datasets/movielens/.

Code Availibility

Most of the work presented in this paper builds on top of open-source libraries. Specifically, the open-source TextWiser library is used for generating item embeddings from text which is available at http://github.com/fidelity/textwiser. Multi-armed bandit recommendation models are built using Mab2Rec which is available at http://github.com/fidelity/mab2rec. The optimization formulations presented in Algorithm 1 is solved using the open-source Python-MIP library, which is available at https://github.com/coin-or/python-mip.

References

  1. Lake, T., Williamson, S.A., Hawk, A.T., Johnson, C.C., Wing, B.P.: Large-scale collaborative filtering with product embeddings. (2019). arXiv:1901.04321

  2. Covington, P., Adams, J., Sargin, E.: Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, September 15–19, 2016, 191–198 (2016)

  3. Wu, C., Alvino, C.V., Smola, A.J., Basilico, J.: Using navigation to improve recommendations in real-time. In: Proceedings of the 10th ACM Conference on Recommender Systems, Boston, MA, USA, September 15–19, 2016, 341–348 (2016)

  4. Quadrana, M., Karatzoglou, A., Hidasi, B., Cremonesi, P.: Personalizing session-based recommendations with hierarchical recurrent neural networks. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, Como, Italy, August 27–31, 2017, 130–137 (2017)

  5. Hansen, C., Hansen, C., Maystre, L., Mehrotra, R., Brost, B., Tomasi, F., Lalmas, M.: Contextual and sequential user embeddings for large-scale music recommendation. In: RecSys 2020: Fourteenth ACM Conference on Recommender Systems, Virtual Event, Brazil, September 22-26, 2020, 53–62 (2020)

  6. Mehrotra, R., Shah, C., Carterette, B.A.: Investigating listeners’ responses to divergent recommendations. In: RecSys 2020: Fourteenth ACM Conference on Recommender Systems, Virtual Event, Brazil, September 22–26, 2020, 692–696 (2020)

  7. Ricci, F., Rokach, L., Shapira, B. (eds.): Recommender Systems Handbook (2015)

  8. Kadioglu, S., Kleynhans, B., Wang, X.: Optimized item selection to boost exploration for recommender systems. In: Proceedings of the 18th International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, Vienna, Austria (2021)

  9. Sparck Jones, K.: A statistical interpretation of term specificity and its application in retrieval. Document Retrieval Systems, 132–142 (1988)

  10. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)

  11. Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32. ICML’14, pp. 1188–1196 (2014)

  12. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 (2018)

  13. Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Proceedings of the 13th International Conference on Neural Information Processing Systems. NIPS’00, pp. 535–541. MIT Press, Cambridge, MA, USA (2000)

  14. Golub, G.H., Reinsch, C.: Singular value decomposition and least squares solutions. In: Linear Algebra, pp. 134–151 (1971)

  15. Beasley, J.E.: An algorithm for set covering problem. European J. Operational Res. 31(1), 85–93 (1987)

    Article  MathSciNet  Google Scholar 

  16. Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Machine Learning, Proceedings of the Twenty-first International Conference (ICML 2004), Banff, Alberta, Canada, July 4-8, 2004. ACM International Conference Proceeding Series, 69 (2004)

  17. Caruana, R., Munson, A., Niculescu-Mizil, A.: Getting the most out of ensemble selection. In: Proceedings of the 6th IEEE International Conference on Data Mining (ICDM 2006), 18-22 December 2006, Hong Kong, China, pp. 828–833 (2006)

  18. Schein, A.I., Popescul, A., Ungar, L.H., Pennock, D.M.: Methods and metrics for cold-start recommendations. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 253–260 (2002)

  19. Amatriain, X., Basilico, J.: Past, present, and future of recommender systems: An industry perspective. In: Proceedings of the 10th ACM Conference on Recommender Systems. RecSys ’16, pp. 211–214 (2016)

  20. Pazzani, M.J., Billsus, D.: Content-based recommendation systems. The Adaptive Web: Methods and Strategies of Web Personalization, 325–341 (2007)

  21. Wang, X., Kadioglu, S.: Bayesian deep learning based explorationexploitation for personalized recommendations. In: 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pp. 1715–1719 (2019). IEEE

  22. Roy, N., McCallum, A.: Toward optimal active learning through sampling estimation of error reduction. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 441–448. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA (2001)

  23. Chan, N.N.: A-optimality for regression designs. J. Math. Anal. Appl. 87(1), 45–50 (1982)

    Article  MathSciNet  Google Scholar 

  24. Rubens, N., Elahi, M., Sugiyama, M., Kaplan, D.: Active learning in recommender systems. In: Recommender Systems Handbook, pp. 809–846 (2015)

  25. Strong, E., Kleynhans, B., Kadioglu, S.: Mabwiser: A parallelizable contextual multi-armed bandit library for python. In: 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI 2019), pp. 885–890 (2019). https://github.com/fidelity/mabwiser

  26. Strong, E., Kleynhans, B., Kadıoğlu, S.: Mabwiser: Parallelizable contextual multi-armed bandits. Int. J. Artif. Intell. Tools 30(04), 1–19 (2021)

  27. Kadoığlu, S., Kleynhans, B.: Building higher-order abstractions from the components of recommender systems. In: Proceedings of the Thirty- Eight AAAI Conference on Artificial Intelligence, February 20–27, 2024, Vancouver, Canada (2024). https://github.com/fidelity/mab2rec

  28. Settles, B.: Active learning literature survey. Technical Report TR-1648. University of Wisconsin-Madison Department of Computer Sciences (2009)

  29. Rashid, A.M., Albert, I., Cosley, D., Lam, S.K., McNee, S.M., Konstan, J.A., Riedl, J.: Getting to know you: learning new user preferences in recommender systems. In: Proceedings of the 7th International Conference on Intelligent User Interfaces, pp. 127–134 (2002)

  30. Huang, Z.: Selectively acquiring ratings for product recommendation. In: Proceedings of the Ninth International Conference on Electronic Commerce, pp. 379–388 (2007)

  31. Cheng, H., Koc, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G., Chai, W., Ispir, M., Anil, R., Haque, Z., Hong, L., Jain, V., Liu, X., Shah, H.: Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, DLRS@RecSys 2016, Boston, MA, USA, September 15, 2016, pp. 7–10 (2016)

  32. Wan, M., McAuley, J.J.: Item recommendation on monotonic behavior chains. In: Proceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, Vancouver, BC, Canada, October 2–7, 2018, pp. 86–94 (2018)

  33. Wan, M., Misra, R., Nakashole, N., McAuley, J.J.: Fine-grained spoiler detection from large-scale review corpora. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pp. 2605–2610 (2019)

  34. Harper, F., Konstan, J.: The movielens datasets: History and context. ACM Trans. Interactive Intell. Syst. 5(4) (2015)

  35. Toffolo, T.A.M., Santos, H.G.: python-mip: Version 1.9.1. https://www.python-mip.com/

  36. Forrest, J.J., Vigerske, S., Santos, H.G., Ralphs, T., Hafer, L., Kristjansson, B., EdwinStraver, Lubin, M., Saltzman, M.: coin-or/Cbc: Version 2.10.5. Zenodo (2020)

  37. Buitinck, L., Louppe, G., Blondel, M., Pedregosa, F., Mueller, A., Grisel, O., Niculae, V., Prettenhofer, P., Gramfort, A., Grobler, J., Layton, R., VanderPlas, J., Joly, A., Holt, B., Varoquaux, G.: API design for machine learning software: experiences from the scikit-learn project. In: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pp. 108–122 (2013)

  38. Kilitçioglu, D., Kadıoğlu, S.: Representing the unification of text featurization using a context-free grammar. In: Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, February 2–9, 2021, Virtual Event (2021). https://github.com/fidelity/textwiser

  39. Vazirani, V.V.: Approximation Algorithms, (2001)

  40. Grave, E., Bojanowski, P., Gupta, P., Joulin, A., Mikolov, T.: Learning word vectors for 157 languages. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018) (2018)

  41. Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

  42. Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715–1725. Association for Computational Linguistics, Berlin, Germany (2016)

  43. van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Machine Learn. Res. 9, 2579–2605 (2008)

    Google Scholar 

  44. Hooker, J.N.: Integrated Methods for Optimization. International series in operations research and management science, vol. 100 (2007)

  45. Koren, Y., Bell, R.M., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)

    Article  Google Scholar 

  46. Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3–6, 2007, pp. 1257–1264 (2007)

  47. Goldberg, D., Nichols, D.A., Oki, B.M., Terry, D.B.: Using collaborative filtering to weave an information tapestry. Commun. ACM 35(12), 61–70 (1992)

    Article  Google Scholar 

  48. Rendle, S.: Factorization machines. In: ICDM 2010, The 10th IEEE International Conference on Data Mining, Sydney, Australia, 14-17 December 2010, pp. 995–1000 (2010)

  49. Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: Deepfm: A factorization-machine based neural network for CTR prediction. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017, pp. 1725–1731 (2017)

  50. Wang, R., Fu, B., Fu, G., Wang, M.: Deep & cross network for ad click predictions. (2017). arXiv:1708.05123

  51. Lops, P., de Gemmis, M., Semeraro, G.: Content-based recommender systems: State of the art and trends. In: Recommender Systems Handbook, pp. 73–105 (2011)

  52. Zhang, S., Yao, L., Sun, A.: Deep learning based recommender system: A survey and new perspectives. (2017). arXiv:1707.07435

  53. McGinty, L., Smyth, B.: On the role of diversity in conversational recommender systems. In: International Conference on Case-Based Reasoning, pp. 276–290 (2003)

  54. Rubens, N., Tomioka, R., Sugiyama, M.: Output divergence criterion for active learning in collaborative settings. Inform. Process. Soc. Japan (IPSJ) Online Trans. 2, 240–249 (2009)

  55. Danziger, S.A., Zeng, J., Wang, Y., Brachmann, R.K., Lathrop, R.H.: Choosing where to look next in a mutation sequence space: Active learning of informative p53 cancer rescue mutants. Bioinformatics 23(13), 104–114 (2007)

    Article  Google Scholar 

  56. Thompson, W.R.: On the Likelihood that one Unknown Probability Exceeds Another in View of the Evidence of Two Samples. Biometrika 25, 285–294 (1933)

    Article  Google Scholar 

  57. Valko, M., Korda, N., Munos, R., Flaounas, I., Cristianini, N.: Finite-time analysis of kernelised contextual bandits. In: Proceedings of the Twenty- Ninth Conference on Uncertainty in Artificial Intelligence, pp. 654–663 (2013)

  58. Cohen, J.: Statistical Power Analysis for the Behavioral Sciences, (2013)

  59. Ryan, T.P., Morgan, J.: Modern experimental design. J. Stat. Theory Practice 1(3–4), 501–506 (2007)

    Article  Google Scholar 

  60. Zhao, L., Pan, S.J., Xiang, E.W., Zhong, E., Lu, Z., Yang, Q.: Active transfer learning for cross-system recommendation. In: Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, July 14-18, 2013, Bellevue, Washington, USA (2013)

  61. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)

    Article  Google Scholar 

Download references

Funding

No funding was received other than the support of the authors’ employer.

Author information

Authors and Affiliations

  1. AI Center of Excellence, Fidelity Investments, Boston, USA

    Serdar Kadıoğlu, Bernard Kleynhans & Xin Wang

  2. Dept. Computer Science, Brown University, Providence, USA

    Serdar Kadıoğlu

Authors
  1. Serdar Kadıoğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernard Kleynhans
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the design of the overall algorithm presented in this paper. The initial study conception and the design of the multi-level framework are composed by S.K. which is later extended to the active learning setting by B.K. and X. W.. The preparation of the material, data collection and analysis were performed by B.K. and X. W.. All authors contributed to the write-up of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Serdar Kadıoğlu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

All authors declare that we express our consent for the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kadıoğlu, S., Kleynhans, B. & Wang, X. Integrating optimized item selection with active learning for continuous exploration in recommender systems. Ann Math Artif Intell (2024). https://doi.org/10.1007/s10472-024-09941-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10472-024-09941-x

Keywords

Mathematics Subject Classification (2010)

Search

Navigation