World Wide Web

, Volume 18, Issue 1, pp 73–109 | Cite as

Robust evaluation of products and reviewers in social rating systems

  • Mohammad Allahbakhsh
  • Aleksandar Ignjatovic
  • Hamid Reza Motahari-Nezhad
  • Boualem Benatallah


Social rating systems are widely used to harvest user feedback and to support making decisions by users on the Web. Web users may try to exploit such systems by posting unfair or false evaluations for fame or profit reasons. Detecting the real rating scores of products as well as the trustworthiness of reviewers is an important and a very challenging problem. Existing approaches use majority-based methods along with temporal analysis and clustering techniques to tackle this problem, but they are vulnerable to massive intelligent collaborative attacks. In this paper, we propose a set of novel algorithms for robust computation of product rating scores and reviewer trust ranks. We introduce a supporting framework consisting of three main components responsible for calculating a robust rating score for product, behavior analysis of reviewers and trust computation for reviewers. We propose a novel algorithm for calculating robust rating scores for products, in presence of unfair reviews. We introduce a method to analyze the reviewing behavior of users by building a vector reflecting three important aspects of reviewers’ behavior. Finally, we combine these behavior factors using a fuzzy inference method to arrive at a final trust score for every reviewer. Extensive evaluation results shows accuracy of our calculated rating and trust scores as well as robustness of our methods against collusive attacks.


Product rating Reviewer trust Iterative voting Collusive attacks 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: Proceedings of the 20th International Conference on Very Large Data Bases, VLDB ’94, pp. 487–499 (1994)Google Scholar
  2. 2.
    Allahbakhsh, M., Ignjatovic, A.: Rating through voting: an iterative method for robust rating. ArXiv e-prints, arXiv:1211.0390 (2012)
  3. 3.
    Allahbakhsh, M., Ignjatovic, A., Benatallah, B., Beheshti, S.-M.-R., Bertino, E., Foo, N.: Reputation management in crowdsourcing systems. In: 2012 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom), pp. 664–671. IEEE (2012)Google Scholar
  4. 4.
    Ayday, E., Hanseung Lee, and Fekri, F.: An iterative algorithm for trust and reputation management. In: IEEE International Symposium on Information Theory, ISIT 2009, pp. 2051–2055, 28 3 July 2009Google Scholar
  5. 5.
    Chen, B.-C., Guo, J., Tseng, B., Yang, J.: User reputation in a comment rating environment. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’11, pp. 159–167. New York, NY, USA, ACM (2011)Google Scholar
  6. 6.
    Chou, C., Ignjatovic, A., Hu, W.: Efficient computation of robust average of compressive sensing data in wireless sensor networks in the presence of sensor faults. IEEE Trans. Parallel Distrib. Syst. 24(8), 1 (2012)Google Scholar
  7. 7.
    Ciccarelli, G., Cigno, R.L.: Collusion in peer-to-peer systems. Comput. Netw. 55(15), 3517–3532 (2011)CrossRefGoogle Scholar
  8. 8.
    Ignjatovic, A., Lee, C. T., Kutay, C., Guo, H., Compton, P.: Computing marks from multiple assessors using adaptive averaging. In: ICEE (2009)Google Scholar
  9. 9.
    Danescu-Niculescu-Mizil, C., Kossinets, G., Kleinberg, J., Lee, L.: How opinions are received by online communities: a case study on helpfulness votes. In: Proceedings of the 18th International Conference on World Wide Web, WWW ’09, pp. 141–150, ACM, New York, NY, USA(2009)Google Scholar
  10. 10.
    De. Alfaro et al.: Reputation systems for open collaboration. Commun. ACM 54, 81–87 (2011)Google Scholar
  11. 11.
    De Kerchove, C., Van Dooren, P.: Iterative filtering for a dynamical reputation system. Arxiv preprint. arXiv:0711.3964 (2007)
  12. 12.
    De Kerchove, C., Van Dooren, P.: Reputation systems and optimization. Siam News, 41(2), 1–3 (2008)Google Scholar
  13. 13.
    de Kerchove, C., Van Dooren, P.: Iterative filtering in reputation systems. SIAM J. Matrix Anal. Appl. 31(4), 1812–1834 (2010)CrossRefzbMATHGoogle Scholar
  14. 14.
    Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world-wide web. Commun. ACM 54, 86–96 (2011)CrossRefGoogle Scholar
  15. 15.
    Feng, Q., Liu, L., Dai, Y.: Vulnerabilities and countermeasures in context-aware social rating services. ACM Trans. Internet Technol. 11(3), 11:1–11:27 (2012)CrossRefGoogle Scholar
  16. 16.
    Flanagin, A.J., Metzger, M.J., Pure, R., Markov, A.: User-generated ratings and the evaluation of credibility and product quality in ecommerce transactions. In: 2011 44th Hawaii International Conference on System Sciences (HICSS), pp. 1–10. IEEE (2011)Google Scholar
  17. 17.
    Ghose, A., Ipeirotis, P.G.: Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Trans. Knowl. Data Eng. 23(10), 1498–1512 (2011)CrossRefGoogle Scholar
  18. 18.
    Harmon, A.: Amazon glitch unmasks war of reviewers. In: NY Times, 14 Feb (2004)Google Scholar
  19. 19.
    Ignjatovic, A., Foo, N., Lee, C.T.: An analytic approach to reputation ranking of participants in online transactions. In: The 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, vol. 01, pp. 587–590. IEEE Computer Society, Washington, DC, USA (2008)Google Scholar
  20. 20.
    Jindal, N., Liu, B., Lim, E.-P.: Finding unusual review patterns using unexpected rules. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, CIKM ’10, pp. 1549–1552. New York, NY, USA, ACM (2010)Google Scholar
  21. 21.
    Kamvar, S.D., Schlosser, M.T., Garcia-Molina, H.: The eigentrust algorithm for reputation management in p2p networks. In: Proceedings of the 12th International Conference on World Wide Web, pp. 640–651. ACM (2003)Google Scholar
  22. 22.
    Laureti, P., Moret, L., Zhang, Y.C., Yu, Y.K.: Information filtering via iterative refinement. EPL (Europhysics Letters) 75, 1006 (2006)CrossRefMathSciNetGoogle Scholar
  23. 23.
    Lee, C.T., Rodrigues, E.M., Kazai, G., Milic-Frayling, N., Ignjatovic, A.: Model for voter scoring and best answer selection in community q&a services. IEEE/WIC/ACM Int. Conf. Web Intelligence and Intelligent Agent Technology, 1, 116–123 (2009)Google Scholar
  24. 24.
    Leskovec, J., Huttenlocher, D., Kleinberg, J.: Predicting positive and negative links in online social networks. In: The 19th International Conference on World Wide Web, pp. 641–650. ACM, New York, NY, USA(2010)Google Scholar
  25. 25.
    Li, R.-H., Yu, J. X., Huang, X., Cheng, H.: Robust reputation-based ranking on bipartite rating networks. In: SDM, pp. 612–623 (2012)Google Scholar
  26. 26.
    Lian, Q., et al.: An empirical study of collusion behavior in the maze p2p file-sharing system. In: Proceedings of the 27th International Conference on Distributed Computing Systems, 56 pp. IEEE Computer Society (2007)Google Scholar
  27. 27.
    Likert, R.: A technique for the measurement of attitudes. Arch. Psychol. 22(140), 55 (1932)Google Scholar
  28. 28.
    Lim, E., et al.: Detecting product review spammers using rating behaviors. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, pp. 939–948. ACM (2010)Google Scholar
  29. 29.
    Liu, Y., Yang, Y., Sun, Y.L.: Detection of collusion behaviors in online reputation systems. In: 2008 42nd Asilomar Conference on Signals, Systems and Computers, pp. 1368–1372. IEEE (2008)Google Scholar
  30. 30.
    Morgan, J., Brown, J.: Reputation in online auctions: The market for trust. Calif. Manag. Rev. 49(1), 61–81 (2006)CrossRefGoogle Scholar
  31. 31.
    Malik, Z., Bouguettaya, A.: Reputation bootstrapping for trust establishment among web services. IEEE Internet Comput. 13(1), 40–47 (2009)CrossRefGoogle Scholar
  32. 32.
    Mendel, J.M.: Fuzzy logic systems for engineering: a tutorial. Proc. IEEE 83(3), 345–377 (1995)CrossRefGoogle Scholar
  33. 33.
    Mukherjee, A., Liu, B., Glance, N.: Spotting fake reviewer groups in consumer reviews. In: Proceedings of the 21st International Conference on World Wide Web, WWW ’12, pp. 191–200. ACM, New York, NY, USA (2012)Google Scholar
  34. 34.
    Murugesan, S.: Understanding web 2.0. IT Professional 9(4), 34–41 (2007)CrossRefGoogle Scholar
  35. 35.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: bringing order to the web. Technical Report 1999–66, Stanford InfoLab. Previous number = SIDL-WP-1999-0120, November (1999)Google Scholar
  36. 36.
    Quinn, A.J., Bederson, B.B.: Human computation: a survey and taxonomy of a growing field. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, pp. 1403–1412, ACM, New York, NY, USA (2011)Google Scholar
  37. 37.
    Schall, D., Skopik, F., Dustdar, S.: Expert discovery and interactions in mixed service-oriented systems. IEEE Trans. Serv. Comput. 5(2), 233–245 (2012)CrossRefGoogle Scholar
  38. 38.
    Song, S., Hwang, K., Macwan, M.: Fuzzy trust integration for security enforcement in grid computing. In: Jin, H., Gao, G.R., Xu, Z., Chen, H. (ed.) Network and Parallel Computing, Lecture Notes in Computer Science, vol. 3222, pp. 9–21. Springer Berlin Heidelberg (2004)Google Scholar
  39. 39.
    Sun, Y., Liu, Y.: Security of online reputation systems: The evolution of attacks and defenses. IEEE Signal Proc. Mag. 29(2), 87–97 (2012)CrossRefGoogle Scholar
  40. 40.
    Swamynathan, G., Almeroth, K., Zhao, B.: The design of a reliable reputation system. Electron. Commer. Res. 10, 239–270 (2010). doi: 10.1007/s10660-010-9064-y CrossRefzbMATHGoogle Scholar
  41. 41.
    Van Leekwijck, W., Kerre, E.E.: Defuzzification: criteria and classification. Fuzzy Sets Syst. 108(2), 159–178 (1999)CrossRefzbMATHGoogle Scholar
  42. 42.
    Wang, G., Wilson, C., Zhao, X., Zhu, Y., Mohanlal, M., Zheng, H., Zhao, B.Y.: Serf and turf: crowdturfing for fun and profit. In Proceedings of the 21st international conference on World Wide Web, WWW ’12, pp. 679–688, ACM, New York, NY, USA (2012)Google Scholar
  43. 43.
    Yang, Y., Feng, Q., Sun, Y.L., Dai, Y.: Reptrap: a novel attack on feedback-based reputation systems. In: Proceedings of the 4th international conference on Security and privacy in communication netowrks, SecureComm ’08, pp. 8:1–8:11, ACM, New York, NY, USA (2008)Google Scholar
  44. 44.
    Yang, Y.-F., Feng, Q.-Y., Sun, Y., Dai, Y.-F.: Dishonest behaviors in online rating systems: cyber competition, attack models, and attack generator. J. Comput. Sci. Technol. 24(5), 855–867 (2009)CrossRefGoogle Scholar
  45. 45.
    Yu, Y.-K., Zhang, Y.-C., Laureti, P., Moret, L.: Decoding information from noisy, redundant, and intentionally distorted sources. Physica A: Statistical Mechanics and its Applications 371(2), 732–744 (2006)CrossRefGoogle Scholar
  46. 46.
    Zhou, Y.-B., Lei, T., Zhou, T.: A robust ranking algorithm to spamming. EPL (Europhysics Letters) 94(4), 48002 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Mohammad Allahbakhsh
    • 1
  • Aleksandar Ignjatovic
    • 1
  • Hamid Reza Motahari-Nezhad
    • 1
  • Boualem Benatallah
    • 1
  1. 1.School of Computer Science and EngineeringUNSWSydneyAustralia

Personalised recommendations