Skip to main content
SpringerLink
Log in

Comprehensive Analysis of Multimodal Recommender Systems

  • Conference paper
  • First Online:
Data Intelligence and Cognitive Informatics

Part of the book series: Algorithms for Intelligent Systems ((AIS))

  • 1120 Accesses

Abstract

Recommender systems are destined to solve the immense issue of analyzing information overload and supporting customers in decision making with more relevant and personalized information. Most of recommender systems only consider the feedback provided by the customers or content of items. They do not consider the different modes of available information. Smartphones, smart devices, web 2.0, etc., enable users to generate different multimedia content which may help to learn about user’s preferences. Multimodal information can be analyzed to learn users’ preference dynamics and generate more accurate personalized information by considering different modes of available information simultaneously. In recent years, multimodal recommender systems have been developed by using multimodal information of users and items. In this paper, a comprehensive analysis of multimodal recommender systems is provided. Our paper focuses on various aspects such as modality, applications and techniques for multimodal recommender systems. The positive and negative aspects of using multimodality are also discussed in recommender systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Iyengar SS, Lepper MR (2000) When choice is demotivating: can one desire too much of a good thing? J Pers Soc Psychol 79(6):995

    Google Scholar 

  2. Ricci F, Rokach L, Shapira B (2015) Recommender systems: introduction and challenges. In: Recommender systems handbook. Springer, Boston, pp 1–34

    Google Scholar 

  3. Shah RR (2016) Multimodal analysis of user-generated content in support of social media applications. In: Proceedings of the ACM on international conference on multimedia retrieval, pp 423–426

    Google Scholar 

  4. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749

    Article  Google Scholar 

  5. Burke R (2002) Hybrid recommender systems: survey and experiments. User Model User-Adap Inter 12(4):331–370

    Article  MATH  Google Scholar 

  6. Yang X, Guo Y, Liu Y, Steck H (2014) A survey of collaborative filtering based social recommender systems. Comput Commun 41:1–10

    Article  Google Scholar 

  7. Milicevic AK, Nanopoulos A, Ivanovic M (2010) Social tagging in recommender systems: a survey of the state-of-the-art and possible extensions. Artif Intell Rev 33(3):187–209

    Article  Google Scholar 

  8. Zhang S, Yao L, Sun A, Tay Y (2019) Deep learning based recommender system: a survey and new perspectives. ACM Comput Surv (CSUR) 52(1):1–38

    Google Scholar 

  9. Cantador I, Fernández-Tobías I, Berkovsky S, Cremonesi, P (2015) Cross-domain recommender systems. In: Recommender systems handbook. Springer, Boston, pp 119–159

    Google Scholar 

  10. Zhang Q, Wu D, Lu J, Liu F, Zhang G (2017) A cross-domain recommender system with consistent information transfer. Decis Support Syst 104:49–63

    Article  Google Scholar 

  11. Kaklauskas A, Gudauskas R, Kozlovas M, Peciure L, Lepkova N, Cerkauskas J, Banaitis A (2016) An affect-based multimodal video recommendation system. Stud Inf Control 25(1):6

    Google Scholar 

  12. Oramas S, Nieto O, Sordo M, Serra X (2017) A deep multimodal approach for cold-start music recommendation. In: Proceedings of the 2nd workshop on deep learning for recommender systems, pp 32–37

    Google Scholar 

  13. Yukawa M, Hayashi Y, Ogawa H, Kryssanov VV (2012) A group recommendation system with a multi-modal user interface. In: The 6th international conference on soft computing and ıntelligent systems, and The 13th international symposium on advanced ıntelligence systems. IEEE, pp 2158–2163

    Google Scholar 

  14. Kuo FF, Shan MK, Lee SY (2013) Background music recommendation for video based on multimodal latent semantic analysis. In: 2013 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

    Google Scholar 

  15. Domingues MA, Gouyon F, Jorge AM, Leal JP, Vinagre J, Lemos L, Sordo M (2013) Combining usage and content in an online recommendation system for music in the long tail. Int J Multimedia Inf Retrieval 2(1):3–13

    Article  Google Scholar 

  16. Tzanetakis G, Cook P (2000) Marsyas: a framework for audio analysis. Organised Sound 4(3):169–175

    Google Scholar 

  17. Da Costa AF, Manzato MG (2016) Exploiting multimodal interactions in recommender systems with ensemble algorithms. Inf Syst 56:120–132

    Article  Google Scholar 

  18. Da Costa AF, Domingues MA, Rezende SO, Manzato MG (2014, Aug) Improving personalized ranking in recommender systems with multimodal interactions. In: 2014 IEEE/WIC/ACM International joint conferences on web intelligence (WI) and intelligent agent technologies (IAT), vol 1. IEEE, pp 198–204

    Google Scholar 

  19. Arapakis I, Moshfeghi Y, Joho H, Ren R, Hannah D, Jose JM (2009) Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. In: IEEE International conference on multimedia and expo, pp 1440–1443

    Google Scholar 

  20. Cho S, Lee M, Jang C, Choi E (2006) Multidimensional filtering approach based on contextual information. In: International conference on hybrid ınformation technology, vol 2. IEEE, pp 497–504

    Google Scholar 

  21. Peska L (2017) Multimodal ımplicit feedback for recommender systems. In: ITAT, pp 240–245

    Google Scholar 

  22. Da Costa AF, Manzato MG (2014) Multimodal interactions in recommender systems: an ensembling approach. In: Brazilian conference on ıntelligent systems. IEEE, pp 67–72

    Google Scholar 

  23. Jia X, Wang A, Li X, Xun G, Xu W, Zhang A (2015) Multi-modal learning for video recommendation based on mobile application usage. In: International conference on big data (big data). IEEE, pp 837–842

    Google Scholar 

  24. Laurier C, Grivolla J, Herrera P (2008) Multimodal music mood classification using audio and lyrics. In: Seventh international conference on machine learning and applications. IEEE, pp 688–693

    Google Scholar 

  25. Yang B, Mei T, Hua XS, Yang L, Yang SQ, Li M (2007) Online video recommendation based on multimodal fusion and relevance feedback. In: Proceedings of the 6th ACM international conference on image and video retrieval, pp 73–80

    Google Scholar 

  26. Campigotto P, Rudloff C, Leodolter M, Bauer D (2016) Personalized and situation-aware multimodal route recommendations: the FAVOUR algorithm. IEEE Trans Intell Transp Syst 18(1):92–102

    Google Scholar 

  27. Rafailidis D, Kefalas P, Manolopoulos Y (2017) Preference dynamics with multimodal user-item interactions in social media recommendation. Expert Syst Appl 74:11–18

    Article  Google Scholar 

  28. Wang X, Zhao YL, Nie L, Gao Y, Nie W, Zha ZJ, Chua TS (2014) Semantic-based location recommendation with multimodal venue semantics. IEEE Trans Multimedia 17(3):409–419

    Article  Google Scholar 

  29. Zhao Z, Yang Q, Lu H, Weninger T, Cai D, He X, Zhuang Y (2017) Social-aware movie recommendation via multimodal network learning. IEEE Trans Multimedia 20(2):430–440

    Article  Google Scholar 

  30. Alhamid MF, Rawashdeh M, El Saddik A (2013) Towards context-aware recommendations of multimedia in an ambient intelligence environment. In: IEEE International symposium on multimedia. IEEE, pp 409–414

    Google Scholar 

  31. Liu L, Gan J (2017) Using multi-modal topic modeling in national culture resources: methods and applications. In: 9th International conference on ıntelligent human-machine systems and cybernetics (IHMSC), vol 2. IEEE, pp 298–301

    Google Scholar 

  32. Huang Z, Xu X, Ni J, Zhu H, Wang C (2009) Multimodal representation learning for recommendation in Internet of Things. IEEE Internet Things J 6(6):10675–10685

    Article  Google Scholar 

  33. Salah A, Truong QT, Lauw HW (2020) Cornac a comparative framework for multimodal recommender systems. J Mach Learn Res 21(95):1–5

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Netaji Subhas Institute of Technology, Delhi, India

    Viomesh Kumar Singh

  2. Netaji Subhas University of Technology, Delhi, India

    Sangeeta Sabharwal

  3. BML Munjal University, Gurgaon, India

    Goldie Gabrani

Authors
  1. Viomesh Kumar Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sangeeta Sabharwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Goldie Gabrani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Viomesh Kumar Singh .

Editor information

Editors and Affiliations

  1. GITAM University, Bangalore, India

    I. Jeena Jacob

  2. Department of Mathematics and Computer Science, Concordia University Chicago, River Forest, IL, USA

    Selvanayaki Kolandapalayam Shanmugam

  3. University of Florida, Gainesville, FL, USA

    Selwyn Piramuthu

  4. Gdańsk University of Technology, Gdańsk, Poland

    Przemyslaw Falkowski-Gilski

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Singh, V.K., Sabharwal, S., Gabrani, G. (2021). Comprehensive Analysis of Multimodal Recommender Systems. In: Jeena Jacob, I., Kolandapalayam Shanmugam, S., Piramuthu, S., Falkowski-Gilski, P. (eds) Data Intelligence and Cognitive Informatics. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-15-8530-2_70

Download citation

Publish with us

Policies and ethics

Search

Navigation