Skip to main content

Comparing context-aware recommender systems in terms of accuracy and diversity

User Modeling and User-Adapted Interaction volume 24, pages 35–65 (2014)Cite this article

Abstract

Although the area of context-aware recommender systems (CARS) has made a significant progress over the last several years, the problem of comparing various contextual pre-filtering, post-filtering and contextual modeling methods remained fairly unexplored. In this paper, we address this problem and compare several contextual pre-filtering, post-filtering and contextual modeling methods in terms of the accuracy and diversity of their recommendations to determine which methods outperform the others and under which circumstances. To this end, we consider three major factors affecting performance of CARS methods, such as the type of the recommendation task, context granularity and the type of the recommendation data. We show that none of the considered CARS methods uniformly dominates the others across all of these factors and other experimental settings; but that a certain group of contextual modeling methods constitutes a reliable “best bet” when choosing a sound CARS approach since they provide a good balance of accuracy and diversity of contextual recommendations.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Notes

  1. The Tidemann-Hall index is very similar to the Rosenbluth index, the only difference between them is that they rank categories differently. The categories are ranked in ascending order in the Rosenbluth index and in descending order in the Tidemann-Hall index. We decided to use the Tidemann-Hall index since our aim was to compare datasets with different categories (Meilak 2008) and the Tidemann-Hall is better suited for this task than the Rosenbluth index because it is more sensitive to the absolute number of categories (the largest category receives a weight equal to one in the formula).

References

  • Adomavicius, G., Tuzhilin, A.: Extending recommender systems: a multidimensional approach, pp. 4–6. IJCAI workshop on intelligent techniques for web personalization, Seattle (2001)

  • Adomavicius, G., Sankaranarayanan, R., Sen, S., Tuzhilin, A.: Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Inf. Syst. 23, 103–145 (2005)

    Article  Google Scholar 

  • Adomavicius, G., Tuzhilin, A.: Context-aware recommender systems’. ACM RecSys Tutorial, pp. 335–336 (2008)

  • Adomavicius, G., Kwon, Y.: Toward more diverse recommendations: Item re-ranking methods for recommender systems. 19th Workshop on information technologies and systems (WITS), Phoenix

  • Adomavicius, G., Mobasher, B., Ricci, F., Tuzhilin, A.: Context-aware recommender systems. AI Mag. 32(3), 67–80 (2011)

    Google Scholar 

  • Adomavicius, G., Tuzhilin, A.: Context-aware recommender systems. In: Handbook on Recommender Systems, pp. 217–253. Springer (2011)

  • Adomavicius, G., Kwon, Y.: Improving aggregate recommendation diversity using ranking-based techniques. IEEE Trans. Knowl. Data Eng. 24(5), 896–911 (2012)

    Google Scholar 

  • Bader, R., Neufeld, E., Woerndl, W., V. Prinz: Context-aware POI recommendations in an automotive scenario using multi-criteria decision making methods. Workshop on context-awareness in retrieval and recommendation, pp. 23–30. Chicago (2011)

  • Baltrunas, L., Amatriain, X.: Towards time-dependant recommendation based on implicit feedback. First workshop on context-aware recommender systems, pp. 1–5. New York (2009)

  • Baltrunas, L., Ludwig, B., Ricci, F.: Matrix factorization techniques for context aware recommendation. Fifth ACM conference on recommender systems, pp. 301–304. Chicago (2011)

  • Baltrunas, L., Ricci, F.: Experimental evaluation of context-dependent collaborative filtering using item splitting. User Modeling and User-Adapted Interaction, Special issue on Context-Aware Recommender Systems (this issue) (2014)

  • Billsus, D., Pazzani, M.: User modeling for adaptive news access. User Model. User Adapt. Interact. 10(2–3), 147–180 (2000)

    Article  Google Scholar 

  • Bracalente, B., Cossignani, M., Mulas, A.: Statistica aziendale. McGraw-Hill Italia, Milano (2009)

    Google Scholar 

  • Campos, P.G., F. Díez and I. Cantador, : Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols. User Modeling and User-Adapted Interaction, Special issue on Context-Aware Recommender Systems (this issue) (2014)

  • Cena, F., Console, L., Gena, C., Goy, A., Levi, G., Modeo, S., Torre, I.: Integrating heterogeneous adaptation techniques to build a flexible and usable mobile tourist guide. AI Commun. 19(4), 369–384 (2006)

    MATH  MathSciNet  Google Scholar 

  • Cremonesi, P., Garza, P., Quintarelli, E., Turrin, R.: Top-N Recommendations on Unpopular Items with Contextual Knowledge, 2011 Workshop on Context-aware Recommender Systems. Chicago (2011)

  • De, A., Desarkar, M.S., Ganguly, N., Mitra, P.: Local learning of item dissimilarity using content and link structure. Sixth ACM conference on recommender systems, pp. 221–224. Dublin (2012)

  • Dourish, P.: What we talk about when we talk about context. Pers. Ubiquitous Comput. 8, 19–30 (2004)

    Article  Google Scholar 

  • Fagà, R., Furtado, B.C., Maximino, F., Cattelan, R.G. da Pimentel, M.G.C.: Context information exchange and sharing in a peer-to-peer community: a video annotation scenario. 27th ACM international conference on design of communication SIGDOC2009, pp. 265–272. Bloomington (2009)

  • Feng, Q., Liu, L., Dai, Y.: Vulnerabilities and countermeasures in context-aware social rating services. Trans. Internet Technol. 11(3), 1–27 (2012)

    Article  Google Scholar 

  • Ge, Y., Liu, Q., Xiong, H., Tuzhilin, A., Chen, J.: Cost-aware travel tour recommendation. 17th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 983–991. San Diego (2011)

  • Gorgoglione, M., Panniello, U.: Including context in a transactional recommender system using a pre-filtering approach: two real E-commerce applications. The 23rd IEEE international conference on advanced information networking and applications (AINA-09), pp. 667–672. Bradford (2009)

  • Gorgoglione, M., Panniello, U., Tuzhilin, A.: The effect of context-aware recommendations on customer purchasing behavior and trust. Fifth ACM conference on recommender systems, pp. 85–92. Chicago (2011)

  • Hariri, N., Mobasher, B., Burke, R.: Context-aware music recommendation based on latenttopic sequential patterns. Sixth ACM conference on recommender systems, pp. 131–138. Dublin (2012)

  • Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5–53 (2004)

    Article  Google Scholar 

  • Hu, R., Pu, P.: Helping users perceive recommendation diversity. Workshop on novelty and diversity in recommender systems, DiveRS. Chicago (2011)

  • Hussein, T., Linder, T., Gaulke, W., Ziegler, J.: Hybreed: a software framework for developing context-aware hybrid recommender systems. User Modeling and User-Adapted Interaction, Special issue on Context-Aware Recommender Systems (this issue) (2014)

  • Kaminskas, M., Ricci, F.: Location-adapted music recommendation using tags. 19th International conference on user modeling, adaption, and personalization, pp. 183–194. Girona (2011)

  • Levi, A., Mokryn, O., Diot, C., Taft, N.: Finding a needle in a haystack of reviews: cold start context-based hotel recommender system. Sixth ACM conference on recommender systems, pp. 115–122. Dublin (2012)

  • Linden, G., Smith, B., York, J.: Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Comput. 7(1), 76–80 (2003)

    Article  Google Scholar 

  • Mahmood, T., Ricci, F., Venturini, A.: Improving recommendation effectiveness by adapting the dialogue strategy in online travel planning. Int. J. Inf. Technol. Tour. 11(4), 285–302 (2010)

    Article  Google Scholar 

  • McDonald, D., Dimmick, J.: The conceptualization and measurement of diversity. Commun. Res. 30(1), 60–79 (2003)

    Article  Google Scholar 

  • Mcginty, L., Smyth, B.: On the role of diversity in conversational recommender systems. Fifth international conference on case-based reasoning, pp. 276–290

  • Meilak, C.: Measuring export concentration: the implications for small states. Bank Valletta Rev. 37, 35–48 (2008)

    Google Scholar 

  • Miller, D.C., Salkind, N.J.: Handbook of Research Design and Social Measurement. Sage Publications, London (2003)

    Google Scholar 

  • Oku, K., Nakajima, S., Miyazaki, J., Uemura, S.: Context-aware SVM for context-dependent information recommendation. 7th International conference on mobile data management, p. 109 (2006)

  • Palmisano, C., Tuzhilin, A., Gorgoglione, M.: Using context to improve predictive modeling of customers in personalization applications. IEEE Trans. Knowl. Data Eng. 20(11), 1535–1549 (2008)

    Article  Google Scholar 

  • Panniello, U., Tuzhilin, A., Gorgoglione, M., Palmisano, C., Pedone, A.: Experimental comparison of pre- vs. post-filtering approaches in context-aware recommender systems. Third ACM conference on recommender systems, pp. 265–268. New York (2009)

  • Panniello, U., Gorgoglione, M.: Incorporating context into recommender systems: an empirical comparison of context-based approaches. Electron. Commer. Res. 12(1), 1–30 (2012)

    Article  Google Scholar 

  • Panniello, U., Tuzhilin, A., Gorgoglione, M.: Comparing context-aware recommender systems in terms of accuracy and diversity: which contextual modeling, pre-filtering and post-filtering methods perform the best. Working paper CBA-12-01. Stern School of Business, NYU (2012)

  • Pareto V.: Cours d’économie politique professé l’université de Lausanne, 3 volumes. Lousanne (1896)

  • Reddy, S., Mascia, J.: Lifetrak: music in tune with your life. 1st ACM international workshop on human-centered multimedia, pp. 25–34. Santa Barbara (2006)

  • Ricci, F.: Mobile recommender systems. Int. J. Inf. Technol. Tour. 12(3), 205–231 (2011)

    Article  Google Scholar 

  • Sae-Ueng, S., Pinyapong, S., Ogino, A., Kato, T.: Personalized shopping assistance service at ubiquitous shop space. 22nd International conference on advanced information networking and applications, pp. 838–843. Los Alamitos (2008)

  • Said, A., Berkovsky, S., De Luca, E. W.: Group recommendation in context. 2nd Challenge on context-aware movie recommendation, pp. 2–4. Chicago (2011)

  • Sharifzadeh, M., Shahabi, C.: The spatial skyline queries. VLDB ’06 Proceedings of the 32nd international conference on very large data bases, pp. 751–762. Seoul (2006)

  • Wang, S.L., Wu, C.T.: Application of context-aware and personalized recommendation to implement an adaptive ubiquitous learning system. Expert Syst. Appl. 38(9), 10831–10838 (2011)

    Article  Google Scholar 

  • Zhang, M., Hurley, N.: Avoiding monotony: improving the diversity of recommendation lists. Second ACM conference on recommender systems. Lausanne (2008)

  • Ziegler, C.N., McNee, S., Konstan, J. Lausen, G. Improving recommendation lists through topic diversification. 14th International conference on world wide web. Chiba (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Mechanics, Management and Mathematics, Politecnico di Bari, Viale Japigia 182, 70126, Bari, BA, Italy

    Umberto Panniello & Michele Gorgoglione

  2. Stern School of Business, New York University, 44 West 4th Street, Room 8-92, New York, NY,  10012, USA

    Alexander Tuzhilin

Authors
  1. Umberto Panniello
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Tuzhilin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michele Gorgoglione
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michele Gorgoglione.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Panniello, U., Tuzhilin, A. & Gorgoglione, M. Comparing context-aware recommender systems in terms of accuracy and diversity. User Model User-Adap Inter 24, 35–65 (2014). https://doi.org/10.1007/s11257-012-9135-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-012-9135-y

Keywords

  • Context-aware recommender systems
  • CARS
  • Pre-filtering
  • Post-filtering
  • Contextual modeling
  • Accuracy
  • Diversity
  • Performance measures