Skip to main content

Neural content-aware collaborative filtering for cold-start music recommendation

Abstract

State-of-the-art music recommender systems are based on collaborative filtering, which builds upon learning similarities between users and songs from the available listening data. These approaches inherently face the cold-start problem, as they cannot recommend novel songs with no listening history. Content-aware recommendation addresses this issue by incorporating content information about the songs on top of collaborative filtering. However, methods falling in this category rely on a shallow user/item interaction that originates from a matrix factorization framework. In this work, we introduce neural content-aware collaborative filtering, a unified framework which alleviates these limits, and extends the recently introduced neural collaborative filtering to its content-aware counterpart. This model leverages deep learning for both extracting content information from low-level acoustic features and for modeling the interaction between users and songs embeddings. The deep content feature extractor can either directly predict the item embedding, or serve as a regularization prior, yielding two variants (strict and relaxed) of our model. Experimental results show that the proposed method reaches state-of-the-art results for both warm- and cold-start music recommendation tasks. We notably observe that exploiting deep neural networks for learning refined user/item interactions outperforms approaches using a more simple interaction model in a content-aware framework.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. While \(\mathbf {r}_i\) denotes the i-th column of \(\mathbf {R}\), \(\mathbf {r}_u\) denotes its u-th row, which might appear as a slight notation abuse. Indeed, using the same notation convention, the u-th row should be denoted by \([\mathbf {R}^{\mathsf {T}}]_u^{\mathsf {T}}\). Nonetheless, we decided to keep the notation \(\mathbf {r}_u\) for brevity.

  2. Note that an alternative common choice consists in considering batches of users instead of items. However, in a content-aware framework, considering batches of items is more straightforward to train both the collaborative filtering part and the content extractor part, since the latter operates on items (and not on users).

  3. https://github.com/magronp/ncacf.

  4. https://pytorch.org/.

  5. http://millionsongdataset.com/tasteprofile/.

  6. https://us.7digital.com/.

  7. For these reasons, we did not consider alternative datasets such as KKBox (https://www.kaggle.com/c/kkbox-music-recommendation-challenge/data), for which only the listening history is directly available.

  8. The Echo Nest developer API (http://developer.echonest.com/) was not accessible at the time of conducting this research.

  9. http://www.ifs.tuwien.ac.at/mir/msd/download.html.

  10. https://zenodo.org/record/3258042#%23.YpYacTnP1so.

References

  • Basbug ME, Engelhardt BE (2016) Hierarchical compound Poisson factorization. In: Proceedings of the 33rd International Conference on Machine Learning (ICML) pp 1795–1803

  • Bertin-Mahieux T, Ellis DP, Whitman B, Lamere B (2011) The million song dataset. In: Proceedings of the 12th International Society for Music Information Retrieval Conference (ISMIR) pp 591–596. https://doi.org/10.7916/D8NZ8J07

  • Bingham E, Kabán A, Fortelius M (2009) The aspect Bernoulli model: Multiple causes of presences and absences. Pattern Anal & Appl 12(1):55–78. https://doi.org/10.1007/s10044-007-0096-4

    MathSciNet  Article  MATH  Google Scholar 

  • Bogdanov D, Wack N, Gómez E, Gulati S, Herrera P, Mayor O, Roma G, Salamon J, Zapata J, Serra X (2013) ESSENTIA: An audio analysis library for music information retrieval. In: Proceedings of the 14th International Conference on Music Information Retrieval (ISMIR) pp 319–326

  • Chen W, Cai F, Chen H, Rijke MD (2019) Joint neural collaborative filtering for recommender systems. ACM Trans on Inf Syst 37(39):1–30. https://doi.org/10.1145/3343117

    Article  Google Scholar 

  • Cheng HT, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, Anderson G, Corrado G, Chai W, Ispir M, Anil R, Haque Z, Hong L, Jain V, Liu X, Shah H (2016) Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems pp 7–10. https://doi.org/10.1145/2988450.2988454

  • Cheng Z, Shen J (2016) On effective location-aware music recommendation. ACM Trans on Inf Syst 34(13):1–32. https://doi.org/10.1145/2846092

    Article  Google Scholar 

  • Chen T, Sun Y, Shi Y, Hong L (2017) On sampling strategies for neural network-based collaborative filtering. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’17) pp 767–776. https://doi.org/10.1145/3097983.3098202

  • Fang Y, Si L (2011) Matrix co-factorization for recommendation with rich side information and implicit feedback. In: Proceedings of the 2nd International Workshop on Information Heterogeneity and Fusion in Recommender Systems (HetRec ’11) pp 65–69. https://doi.org/10.1145/2039320.2039330

  • Ferwerda B, Yang E, Schedl M, Tkalcic M (2015) Personality traits predict music taxonomy preferences. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15) pp 2241–2246. https://doi.org/10.1145/2702613.2732754

  • Flexer A (2006) Statistical evaluation of music information retrieval experiments. J of New Music Res 35(2):113–120. https://doi.org/10.1080/09298210600834946

    Article  Google Scholar 

  • Gillhofer M, Schedl M (2015) Iron Maiden while jogging, Debussy for dinner? an analysis of music listening behavior in context. In: Proceedings of the 21st International conference on MultiMedia Modeling (MMM 2015

  • Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. Proc of the Fourteenth Int Conf on Artificial Intell and Statistics 15:315–323

    Google Scholar 

  • Gopalan PK, Charlin L, Blei D (2014a) Content-based recommendations with Poisson factorization. In: Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14) pp 3176–3184

  • Gopalan PK, Hofman JM, Blei D (2014b) Scalable recommendation with hierarchical Poisson factorization. In: Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence (UAI’15) pp 326–335

  • Gouvert O, Oberlin T, Févotte C (2018) Matrix co-factorization for cold-start recommendation. In: Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR) pp 792–798

  • Gouvert O, Oberlin T, Févotte C (2019) Recommendation from raw data with adaptive compound Poisson factorization. In: Proceedings of the International Conference on Machine Learning (ICML)

  • He X, Du1 X, Wang X, Tian F, Tang J, Chua TS (2018) Outer product-based neural collaborative filtering. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI-18) pp 2227–2233

  • He X, Liao L, Zhang H, Nie L, Hu X, Chua TS (2017) Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web (WWW ’17) pp 173–182. https://doi.org/10.1145/3038912.3052569

  • He X, Zhang H, Kan MY, Chua TS (2016b) Fast matrix factorization for online recommendation with implicit feedback. In: Proceedings of the 39th International ACM Conference on Research and Development in Information Retrieval (SIGIR ’16) pp 549–558. https://doi.org/10.1145/2911451.2911489

  • He K, Zhang X, Ren S, Sun J (2016a) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  • Hsieh CK, Yang L, Cui Y, Lin TY, Belongie S, Estrin D (2017) Collaborative metric learning. In: Proceedings of the 26th International Conference on World Wide Web (WWW ’17) pp 193–201. https://doi.org/10.1145/3038912.3052639

  • Huan H, Wei Z, Liang L, Yang L (2017) Collaborative filtering recommendation model based on convolutional denoising auto encoder. In: Proceedings of the 12th Chinese Conference on Computer Supported Cooperative Work and Social Computing (ChineseCSCW ’17) pp 64–71. https://doi.org/10.1145/3127404.3127420

  • Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: Proceedings of the 2008 Eighth IEEE International Conference on Data Mining (ICDM ’08) pp 263–272. https://doi.org/10.1109/ICDM.2008.22

  • Jeunen O, Van Balen J, Goethals B (2020) Closed-form models for collaborative filtering with side-information. In: Proceedings of the 14th ACM Conference on Recommender Systems (RecSys ’20) pp 651–656. https://doi.org/10.1145/3383313.3418480

  • Kim T, Lee J, Nam J (2018) Sample-level CNN architectures for music auto-tagging using raw waveforms. In: Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp 366–370

  • Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the International Conference on Learning Representations (ICLR)

  • Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Comput 42(8):30–37. https://doi.org/10.1109/MC.2009.263

    Article  Google Scholar 

  • Laplante A (2014) Improving music recommender systems: What can we learn from research on music tastes? In: Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR) pp 451–456

  • LeCun YA, Bottou L, Orr GB, Müller KR (2012) Efficient BackProp, Springer Berlin Heidelberg, Berlin, Heidelberg pp 9–48. https://doi.org/10.1007/978-3-642-35289-8_3

  • Lee J, Lee K, Park J, Park J, Nam J (2018) Deep content-user embedding model for music recommendation. arXiv: 1807.06786

  • Liang D, Charlin L, McInerney J, Blei DM (2016) Modeling user exposure in recommendation. In: Proceedings of the International World Wide Web Conference (WWW) pp 951–961. https://doi.org/10.1145/2872427.2883090

  • Liang D, Krishnan RG, Hoffman MD, Jebara T (2018) Variational autoencoders for collaborative filtering. In: Proceedings of the 2018 World Wide Web Conference, WWW ’18 pp 689–698. https://doi.org/10.1145/3178876.3186150

  • Liang D, Zhan M, Ellis DP (2015) Content-aware collaborative music recommendation using pre-trained neural networks. In: Proceedings of the 16th International Society for Music Information Retrieval Conference (ISMIR)

  • Lian J, Zhang F, Xie X, Sun G (2017) CCCFNet: A content-boosted collaborative filtering neural network for cross domain recommender systems. In: Proceedings of the 26th International Conference on World Wide Web Companion pp 817–818. https://doi.org/10.1145/3041021.3054207

  • Lidy T, Rauber A (2005) Evaluation of feature extractors and psycho-acoustic transformations for music genre classification. In: Proceedings of the 6th International Society for Music Information Retrieval Conference (ISMIR)

  • Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM ’15) pp 811–820. https://doi.org/10.1145/2806416.2806527

  • Li X, She J (2017) Collaborative variational autoencoder for recommender systems. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’17) pp 305–314. https://doi.org/10.1145/3097983.3098077

  • Liu X, Ouyang Y, Rong W, Xiong Z (2015) Item category aware conditional restricted boltzmann machine based recommendation. In: Proceeings, Part II, of the 22nd International Conference on Neural Information Processing - Volume 9490, ICONIP 2015 pp 609–616. https://doi.org/10.1007/978-3-319-26535-3_69

  • Li Z, Xu Q, Jiang Y, Cao X, Huang Q (2020) Quaternion-based knowledge graph network for recommendation. In: Proceedings of the 28th ACM International Conference on Multimedia, MM ’20 pp 880–888. https://doi.org/10.1145/3394171.3413992

  • Magron P, Févotte C (2021) Leveraging the structure of musical preference in content-aware music recommendation. In: arXiv: 2010.10276

  • Marlin B, Zemel RS (2004) The multiple multiplicative factor model for collaborative filtering. In: Proceedings of the twenty-first international conference on Machine learningProceedings of the International Conference on Machine Learning (ICML ’04). https://doi.org/10.1145/1015330.1015437

  • Oramas S, Nieto O, Sordo M, Serra X (2017) A deep multimodal approach for cold-start music recommendation. In: Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems (DLRS 2017) pp 32–37. https://doi.org/10.1145/3125486.3125492

  • Pichl M, Zangerle E (2021) User models for multi-context-aware music recommendation. Multimed Tools and Appl 80:22509–22531. https://doi.org/10.1007/s11042-020-09890-7

    Article  Google Scholar 

  • Pons J, Nieto O, Prockup M, Schmidt E, Ehmann A, Serra X (2018) End-to-end learning for music audio tagging at scale. In: Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR) pp 637–644

  • Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI ’09) pp 452–461

  • Rendle S, Krichene W, Zhang L, Anderson J (2020) Neural collaborative filtering vs. matrix factorization revisited. In: Proceedings of the Fourteenth ACM Conference on Recommender Systems (RecSys ’20) pp 240–248. https://doi.org/10.1145/3383313.3412488

  • Salakhutdinov R, Mnih A (2007) Probabilistic matrix factorization. In: Proceedings of the 20th International Conference on Neural Information Processing Systems (NIPS’07) pp 1257–1264

  • Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using markov chain monte carlo. In: Proceedings of the 25th international conference on Machine learning (ICML ’08) pp 880–887. https://doi.org/10.1145/1390156.1390267

  • Salakhutdinov R, Mnih A, Hinton G (2007) Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the International Conference on Machine Learning (ICML) pp 791–798. https://doi.org/10.1145/1273496.1273596

  • Schedl M, Zamani H, Chen CW, Deldjoo Y, Elahi M (2018) Current challenges and visions in music recommender systems research. Int J of Multimed Inf Retr 7(2):95–116. https://doi.org/10.1007/s13735-018-0154-2

    Article  Google Scholar 

  • Schedl M, Knees P, Gouyon F (2017) New paths in music recommender systems research. In: Proceedings of the Eleventh ACM Conference on Recommender Systems (RecSys ’17) pp 392–393. https://doi.org/10.1145/3109859.3109934

  • Schedl M, Knees P, McFee B, Bogdanov D, Kaminskas M (2015) Music recommender systems. In: Recommender Systems Handbook, Springer pp 453–492

  • Schein AI, Popescul A, Ungar LH, Pennock DM (2002) Methods and metrics for cold-start recommendations. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’02 pp 253–260. https://doi.org/10.1145/564376.564421

  • Schindler A, Mayer R, Rauber A (2012) Facilitating comprehensive benchmarking experiments on the million song dataset. In: Proceedings of the 13th International Society for Music Information Retrieval Conference (ISMIR) pp 469–474

  • Schindler A, Rauber A (2012) Capturing the temporal domain in Echonest features for improved classification effectiveness. In: Proceedings of the International Workshop on Adaptive Multimedia Retrieval (AMR) pp 214–227. https://doi.org/10.1007/978-3-319-12093-5_13

  • Soleymani M, Aljanaki A, Wiering F, Veltkamp RC (2015) Content-based music recommendation using underlying music preference structure. In: Proceedings IEEE International Conference on Multimedia and Expo (ICME). https://doi.org/10.1109/ICME.2015.7177504

  • Tran VA, Hennequin R, Royo-Letelier J, Moussallam M (2019) Improving collaborative metric learning with efficient negative sampling. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR’19 pp 1201–1204. https://doi.org/10.1145/3331184.3331337

  • Van den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS’13 pp 2643–2651

  • Wang C, Blei DM (2011) Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD ’11) pp 448–456. https://doi.org/10.1145/2020408.2020480

  • Wang X, He X, Cao Y, Liu M, Chua TS (2019) Kgat: Knowledge graph attention network for recommendation. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’19)

  • Wang Z, Lin G, Tan H, Chen Q, Liu X (2020) CKAN: Collaborative Knowledge-Aware Attentive Network for Recommender Systems. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20 pp 219–228. https://doi.org/10.1145/3397271.3401141

  • Wang H, Shi X, Yeung DY (2016) Collaborative recurrent autoencoder: Recommend while learning to fill in the blanks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS’16) pp 415–423

  • Wang X, Wang Y (2014) Improving content-based and hybrid music recommendation using deep learning. In: Proceedings of the 22nd ACM international conference on Multimedia (MM ’14) pp 627–636. https://doi.org/10.1145/2647868.2654940

  • Wang H, Wang N, Yeung DY (2015) Collaborative deep learning for recommender systems. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’15) pp 1235–1244. https://doi.org/10.1145/2783258.2783273

  • Wang Y, Wang L, Yuanzhi L, He D, Liu TY (2013) A theoretical analysis of NDCG type ranking measures. In: Proceedings of the 26th Conference on Learning Theory (COLT) pp 25–54

  • Wu Y, DuBois C, Zheng AX, Ester M (2016) Collaborative denoising auto-encoders for top-N recommender systems. In: Proceedings of the Ninth ACM International Conference on Web Search and Data Mining (WSDM ’16) pp 153–162. https://doi.org/10.1145/2835776.2835837

  • Xue HJ, Dai X, Zhang J, Huang S, Chen J (2017) Deep matrix factorization models for recommender systems. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17) pp 3203–3209

  • Xue F, He X, Wang X, Xu J, Liu K, Hong R (2019) Deep item-based collaborative filtering for top-N recommendation. ACM Trans on Inf Syst 37(3):1–25. https://doi.org/10.1145/3314578

    Article  Google Scholar 

  • Yoshii K, Goto M, Komatani K, Ogata T, Okuno HG (2006) Hybrid collaborative and content-based music recommendation using probabilistic model with latent user preferences. In: Proceedings of the 7th International Society for Music Information Retrieval Conference (ISMIR)

  • Zangerle E, Pichl M, Schedl M (2020) User models for culture-aware music recommendation: Fusing acoustic and cultural cues. Trans of the Int Soc for Music Inf Retr (TISMIR) 3(1):1–16. https://doi.org/10.5334/tismir.37/

    Article  Google Scholar 

  • Zhang S, Yao L, Sun A, Tay Y (2019) Deep learning based recommender system: A survey and new perspectives. ACM Comput Surveys 52(1):1–38. https://doi.org/10.1145/3285029

    Article  Google Scholar 

  • Zheng L, Noroozi V, Yu PS (2017) Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (WSDM ’17) pp 425–434. https://doi.org/10.1145/3018661.3018665

  • Zuo Y, Zeng J, Gong M, Jiao L (2016) Tag-aware recommender systems based on deep neural networks. Neurocomputing 204:51–60. https://doi.org/10.1016/j.neucom.2015.10.134

    Article  Google Scholar 

Download references

Acknowledgements

The work of P. Magron was conducted while he was with IRIT, Université de Toulouse, CNRS, Toulouse, France. This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme under grant agreement No 681839 (project FACTORY). Experiments presented in this paper were carried out using the Grid’5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Paul Magron.

Additional information

Responsible editor: Johannes Fürnkranz.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Magron, P., Févotte, C. Neural content-aware collaborative filtering for cold-start music recommendation. Data Min Knowl Disc (2022). https://doi.org/10.1007/s10618-022-00859-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10618-022-00859-8

Keywords

  • Content-aware recommendation
  • Neural collaborative filtering
  • Matrix factorization
  • Cold-start problem