User Modeling and User-Adapted Interaction

, Volume 24, Issue 3, pp 219–260 | Cite as

A comparative study of collaboration-based reputation models for social recommender systems

  • Kevin McNally
  • Michael P. O’Mahony
  • Barry Smyth
Original Paper


Today, people increasingly leverage their online social networks to discover meaningful and relevant information, products and services. Thus, the ability to identify reputable online contacts with whom to interact has become ever more important. In this work we describe a generic approach to modeling user and item reputation in social recommender systems. In particular, we show how the various interactions between producers and consumers of content can be used to create so-called collaboration graphs, from which the reputation of users and items can be derived. We analyze the performance of our reputation models in the context of the HeyStaks social search platform, which is designed to complement mainstream search engines by recommending relevant pages to users based on the past experiences of search communities. By incorporating reputation into the existing HeyStaks recommendation framework, we demonstrate that the relevance of HeyStaks recommendations can be significantly improved based on data recorded during a live-user trial of the system.


Reputation Social recommender systems Collaboration graphs 



This work is supported by Science Foundation Ireland under grant 07/CE/I1147.


  1. Amershi, S., Morris, M.R.: Cosearch: a system for co-located collaborative web search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08), Florence, Italy, pp. 1647–1656. ACM, New York (2008)Google Scholar
  2. Aral, E., Muchnik, L., Sundararajan, A.: Distinguishing influence-based contagion from homophily-driven diffusion in dynamic networks. Proc. Natl. Acad. Sci. 106, 21544–21549 (2009)CrossRefGoogle Scholar
  3. Avesani, P., Massa, P., Tiella, R.: A trust-enhanced recommender system application: moleskiing. In: Proceedings of the 2005 ACM Symposium on Applied Computing (SAC ’05), Santa Fe, NM, pp. 1589–1593. ACM, New York (2005)Google Scholar
  4. Bailey, T.L., Gribskov, M.: Combining evidence using p-values: application to sequence homology searches. Bioinformatics 14, 48–54 (1998)CrossRefGoogle Scholar
  5. Bakshy, E., Hofman, J.M., Mason, W.A., Watts, D.J.: Everyone’s an influencer: quantifying influence on Twitter. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining (WSDM ’11), Hong Kong, China, pp. 65–74. ACM, New York (2011)Google Scholar
  6. Basu, C., Hirsh, H., Cohen, W.: Recommendation as classification: using social and content-based information in recommendation. In: Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence (AAAI/IAAI ’98), Madison, WI, USA, pp. 714–720. American Association for Artificial Intelligence, Palo Alto (1998)Google Scholar
  7. Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. In: Proceedings of the 7th International Conference on World Wide Web (WWW ’98), Brisbane, Australia, pp. 107–117. ACM, New York (1998)Google Scholar
  8. Cai, K., Bao, S., Yang, Z., Tang, J., Ma, R., Zhang, L., Su, Z.: OOLAM: an opinion oriented link analysis model for influence persona discovery. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining (WSDM ’11), Hong Kong, China, pp. 645–654. ACM, New York (2011)Google Scholar
  9. Canini, K., Suh, B., Pirolli, P.: Finding credible information sources in social networks based on content and social structure. In: Proceedings of the Third IEEE International Conference on Social Computing (SocialCom ’11), Boston, MA, USA, pp. 1–8 (2011)Google Scholar
  10. Chatterjee, K., de Alfaro, L., Pye, I.: Robust content-driven reputation. In: Proceedings of the 1st ACM Workshop on Artificial Intelligence and Security (AISec ’08), Alexandria, VA, USA, pp. 33–42. ACM, New York (2008)Google Scholar
  11. Cheng, R., Vassileva, J.: Adaptive reward mechanism for sustainable online learning community. In: Proceedings of the 2005 Conference on Artificial Intelligence in Education (AIED ’05), Amsterdam, The Netherlands, pp. 152–159. IOS Press, Amsterdam (2005)Google Scholar
  12. De Alfaro, L., Kulshreshtha, A., Pye, I., Adler, B.T.: Reputation systems for open collaboration. Commun. ACM 54, 81–87 (2011)CrossRefGoogle Scholar
  13. Dellarocas, C.: The digitization of word of mouth: promise and challenges of online feedback mechanisms. Manag. Sci. 49, 1407–1424 (2003)CrossRefGoogle Scholar
  14. Duan, Y., Jiang, L., Qin, T., Zhou, M., Shum, H.Y.: An empirical study on learning to rank of Tweets. In: Proceedings of the 23rd International Conference on Computational Linguistics (COLING ’10), Beijing, China, pp. 295–303. Association for Computational Linguistics (2010)Google Scholar
  15. Gambetta, D., et al.: Can we trust trust? In: Gambetta, D. (ed.) Trust: Making and Breaking Cooperative Relations, pp. 213–237. University of Oxford, Oxford (2000)Google Scholar
  16. Goh, D., Ang, R., Lee, C., Chua, A.: Fight or unite: investigating game genres for image tagging. J. Am. Soc. Inf. Sci. Technol. 62(7), 1311–1324 (2011)CrossRefGoogle Scholar
  17. Golbeck, J.: Generating predictive movie recommendations from trust in social networks. In: Proceedings of the 4th International Conference on Trust Management (iTrust’06), Pisa, Italy, pp. 93–104. Springer, Berlin (2006)Google Scholar
  18. Golbeck, J., Hendler, J.: Accuracy of metrics for inferring trust and reputation in semantic web-based social networks. In: Proceedings of the International Conference on Knowledge Engineering and Knowledge Management (EKAW ’04), Whittlebury Hall, Northamptonshire, UK (2004)Google Scholar
  19. Guha, R., Kumar, R., Raghavan, P., Tomkins, A.: Propagation of trust and distrust. In: Proceedings of the 13th International Conference on World Wide Web (WWW ’04), New York, NY, USA, pp. 403–412. ACM, New York (2004)Google Scholar
  20. Guy, I., Perer, A., Daniel, T., Greenshpan, O., Turbahn, I.: Guess who?: enriching the social graph through a crowdsourcing game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’11), Vancouver, Canada, pp. 1373–1382. ACM, New York (2011)Google Scholar
  21. Hacker, S., von Ahn, L.: Matchin: eliciting user preferences with an online game. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09), Boston, MA, USA, pp. 1207–1216. ACM, New York (2009)Google Scholar
  22. Joinson, A.N.: Looking at, looking up or keeping up with people?: motives and use of Facebook. In: Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (CHI ’09), Florence, Italy, pp. 1027–1036. ACM, New York (2008)Google Scholar
  23. Jones, S., Wilikens, M., Morris, P., Masera, M.: Trust requirements in e-business. Commun. ACM 43(12), 81–87 (2000)CrossRefGoogle Scholar
  24. Jøsang, A., Golbeck, J.: Challenges for robust trust and reputation systems. In: 5th International Workshop on Security and Trust Management (STM ’09), Saint Malo, France. Springer LNCS, New York (2009)Google Scholar
  25. Jøsang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decis. Support Syst. 43(2), 618–644 (2007)CrossRefGoogle Scholar
  26. Keane, M.T., O’Brien, M., Smyth, B.: Are people biased in their use of search engines? Commun. ACM 51, 49–52 (2008)CrossRefGoogle Scholar
  27. Kempe, D., Kleinberg, J., Tardos, E.: Maximizing the spread of influence through a social network. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’03), Washington, DC, USA, pp. 137–146. ACM, New York (2003)Google Scholar
  28. Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. J. ACM 46(5), 604–632 (1999)CrossRefzbMATHMathSciNetGoogle Scholar
  29. Kuter, U., Golbeck, J.: SUNNY: a new algorithm for trust inference in social networks using probabilistic confidence models. In: Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI ’07), Vancouver, Canada, pp. 1377–1382. AAAI Press, Palo Alto (2007)Google Scholar
  30. Kuter, U., Golbeck, J.: Using probabilistic confidence models for trust inference in web-based social networks. ACM Trans. Internet Technol. 10(2), 1–23 (2010)CrossRefGoogle Scholar
  31. Lam, S.K., Riedl, J.: Shilling recommender systems for fun and profit. In: WWW ’04: Proceedings of the 13th International World Wide Web Conference, pp. 393–402. ACM, New York (2004)Google Scholar
  32. Langville, A., Meyer, C.: A survey of eigenvector methods for web information retrieval. Soc. Ind. Appl. Math. Rev. 47(1), 135–161 (2005)zbMATHMathSciNetGoogle Scholar
  33. Law, E., von Ahn, L.: Input-agreement: a new mechanism for collecting data using human computation games. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09), Boston, USA, pp. 1197–1206. ACM, New York (2009)Google Scholar
  34. Lazzari, M.: An experiment on the weakness of reputation algorithms used in professional social networks: the case of Naymz. In: IADIS International Conference e-Society, Porto, Portugal, pp. 519–522 (2010)Google Scholar
  35. Li, H., Bhowmick, S.S., Sun, A.: Casino: towards conformity-aware social influence analysis in online social networks. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pp. 1007–1012. ACM, New York, NY, USA, CIKM ’11 (2011). doi: 10.1145/2063576.2063721
  36. Lih, A.: Wikipedia as participatory journalism: reliable sources? Metrics for evaluating collaborative media as a news resource. In: Proceedings of the 5th International Symposium on Online, Journalism, pp. 16–17 (2004)Google Scholar
  37. Massa, P., Avesani, P.: Trust-aware recommender systems. In: RecSys ’07: Proceedings of the 2007 ACM Conference on Recommender Systems, pp. 17–24. ACM, New York (2007)Google Scholar
  38. Massa, P., Bhattacharjee, B.: Using trust in recommender systems: an experimental analysis. In: Proceedings of the 2nd International Conference on Trust Management (iTrust ’04), pp. 221–235 (2004)Google Scholar
  39. Mayer, R., Davis, J., Schoorman, F.: An integrative model of organizational trust. Acad. Manag. Rev. 20, 709–734 (1995)Google Scholar
  40. McKnight, D., Chervany, N.: The meanings of trust. Technical Report WP9604, University of Minnesota Management Information Research Center (1996)Google Scholar
  41. McNally, K., O’Mahony, M.P., Smyth, B., Coyle, M., Briggs, P.: Towards a reputation-based model of social web search. In: Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI ’10), Hong Kong, China, pp. 179–188. ACM, New York (2010)Google Scholar
  42. McNally, K., O’Mahony, M.P., Smyth, B., Coyle, M., Briggs, P.: A case-study of collaboration and reputation in social web search. ACM Trans. Intell. Syst. Technol. 3(1), 4:1–4:29 (2011)Google Scholar
  43. Mobasher, B., Burke, R., Bhaumik, R., Williams, C.: Toward trustworthy recommender systems: an analysis of attack models and algorithm robustness. ACM Trans. Internet Technol. 7(4), 1–40 (2007)CrossRefGoogle Scholar
  44. Morris, M.R., Horvitz, E.: \(\text{ S }^{{\rm 3}}\): storable, shareable search. In: Proceedings of Human–Computer Interaction–INTERACT, 11th IFIP TC 13 International Conference, Rio de Janeiro, Brazil, pp. 120–123. Springer, New York (2007a)Google Scholar
  45. Morris, M.R., Horvitz, E.: SearchTogether: an interface for collaborative web search. In: Proceedings of the 21st ACM Symposium on User Interface Software and Technology (UIST ’07), Newport, Rhode Island, USA, pp. 3–12. ACM, New York (2007b)Google Scholar
  46. Mui, L., Mohtashemi, M., Halberstadt, A.: A computational model of trust and reputation. In: Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS ’02), Big Island, Hawaii, USA, pp. 2431–2439. IEEE, Palo Alto (2002)Google Scholar
  47. O’Donovan, J.: Capturing trust in social web applications. In: Golbeck, J. (ed.) Computing with Social Trust, pp. 213–257. Springer, New York (2009)CrossRefGoogle Scholar
  48. O’Donovan, J., Smyth, B.: Trust in recommender systems. In: Proceedings of the 10th International Conference on Intelligent User Interfaces (IUI ’05), San Diego, CA, USA, pp. 167–174. ACM, New York (2005)Google Scholar
  49. Olson, J.S., Olson, G.M.: i2i Trust in e-commerce. Commun. ACM 43(12), 41–44 (2000)CrossRefGoogle Scholar
  50. O’Mahony, M.P., Hurley, N.J., Silvestre, G.C.M.: Promoting recommendations: an attack on collaborative filtering. In: Proceedings of the 13th International Conference on Database and Expert Systems Applications (DEXA ’02). Aix en Provence, France, pp. 494–503. Springer, Aix-en-Provence (2002)Google Scholar
  51. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank citation ranking: bringing order to the web. Technical, Report (1999)Google Scholar
  52. Pal, A., Counts, S.: Identifying topical authorities in microblogs. In: Proceedings of the Fourth ACM International Conference on Web Search and Data Mining (WSDM ’11), Hong Kong, China, pp. 45–54. ACM, New York (2011)Google Scholar
  53. Pariser, E.: The Filter Bubble: What the Internet is Hiding From You. Penguin Press HC, New York (2011)Google Scholar
  54. Phelan, O., McCarthy, K., Smyth, B.: Yokie: A curated, real-time, search and discovery system using Twitter. In: 3rd Workshop on Recommender Systems and the Social Web, in association with the 5th ACM Conference on Recommender Systems (RecSys 2011) (2011)Google Scholar
  55. Preston, R., Preston, S.: The Official Biggest Pub Quiz Book Ever!. Carlton Books Ltd, London (2007)Google Scholar
  56. Raub, W., Weesie, J.: Reputation and efficiency in social interactions: an example of network effects. Am. J. Sociol. 96, 626–654 (1990)CrossRefGoogle Scholar
  57. Recuero, R., Araujo, R., Zago, G.: How does social capital affect retweets? In: Adamic, L.A., Baeza-Yates, R.A., Counts, S. (eds.) The Fifth International AAAI Conference on Weblogs and Social Media (ICWSM ’11), Barcelona, Spain. AAAI Press, Palo Alto (2011)Google Scholar
  58. Resnick, P., Zeckhauser, R.: Trust among strangers in Internet transactions: empirical analysis of eBay’s reputation system. Adv. Appl. Microecon. 11, 127–157 (2002)CrossRefGoogle Scholar
  59. Resnick, P., Zeckhauser, R., Friedman, E., Kuwabara, K.: Reputation systems: facilitating trust in Internet interactions. Commun. ACM 43(12), 45–48 (2000)CrossRefGoogle Scholar
  60. Rousseau, D., Sitkin, S., Burt, R., Camerer, C.: Not so different after all: a cross-discipline view of trust. Acad. Manag. Rev. 23(3), 393–404 (1998)CrossRefGoogle Scholar
  61. Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983)zbMATHGoogle Scholar
  62. Schaal, M., Fidan, G., Müller, R.M., Dagli, O.: Quality assessment in the blog space. Learn. Organ. 17(6), 529–536 (2010)CrossRefGoogle Scholar
  63. Shafer, G.: The combination of evidence. Int. J. Intell. Syst. 1(3), 155–179 (1986)CrossRefzbMATHGoogle Scholar
  64. Shneiderman, B.: Designing trust into online experiences. Commun. ACM 43(12), 57–59 (2000)CrossRefGoogle Scholar
  65. Siorpaes, K., Hepp, M.: Games with a purpose for the semantic web. Intell. Syst. 23(3), 50–60 (2008)CrossRefGoogle Scholar
  66. Smyth, B.: A community-based approach to personalizing web search. IEEE Comput. 40(8), 42–50 (2007)Google Scholar
  67. Smyth, B., Briggs, P., Coyle, M., O’Mahony, M.P.: Google shared. A case-study in social search. In: The 17th conference on User Modeling, Adaptation, and Personalization (UMAP ’09), pp. 283–294. Springer, Trento (2009)Google Scholar
  68. Speretta, M., Gauch, S.: Personalized search based on user search histories. In: Proceedings of the 2005 IEEE/WIC/ACM International Conference on Web Intelligence (WI ’05), pp. 622–628. IEEE Computer Society, Washington, DC (2005)Google Scholar
  69. von Ahn, L.: Games with a purpose. IEEE Comput. Mag. 39, 96–98 (2006)Google Scholar
  70. Voorbraak, F.: Combining unreliable pieces of evidence. Technical Report, University of Amsterdam 1995)Google Scholar
  71. Walsh, G., Golbeck, J.: Curator: A game with a purpose for collection recommendation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10), Atlanta, GA, USA, pp. 2079–2082. ACM, New York (2010)Google Scholar
  72. Weng, J., Lim, E.P., Jiang, J., He, Q.: TwitterRank: finding topic-sensitive influential Twitterers. In: Proceedings of the Third ACM International Conference on Web Search and Data Mining (WSDM ’10), New York, NY, USA, pp. 261–270. ACM, New York (2010)Google Scholar
  73. Wu, J.J., Tsang, A.S.L.: Factors affecting members’ trust belief and behaviour intention in virtual communities. Behav. Inf. Technol. 27, 115–125 (2008)CrossRefGoogle Scholar
  74. Yang, J., Adamic, L.A., Ackerman, M.S.: Competing to share expertise: the Taskcn Knowledge Sharing Community. In: Proceedings of the 2nd AAAI Conference on Weblogs and Social Media (ICWSM ’2008), Seattle, WA, USA (2008)Google Scholar
  75. Zeng, H., Alhossaini, M.A., Ding, L., Fikes, R., McGuinness, D.L.: Computing trust from revision history. In: Proceedings of the 2006 International Conference on Privacy, Security and Trust (PST ’06), Markham, ON, Canada, pp. 8:1–8:1. ACM, New York (2006)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  • Kevin McNally
    • 1
  • Michael P. O’Mahony
    • 1
  • Barry Smyth
    • 1
  1. 1.CLARITY Centre for Sensor Web Technologies, School of Computer Science and InformaticsUniversity College DublinDublinIreland

Personalised recommendations