Skip to main content

Estimating global opinions by keeping users from fraud in online review systems

Knowledge and Information Systems volume 55, pages 467–491 (2018)Cite this article

Abstract

In this work, we focus on online review systems, in which users provide opinions about a set of entities (movies, restaurants, etc.) based on their experiences and in turn can check what others prefer. These systems have been proved to be sensitive to fraud and have shown some shortcomings as a result of capturing opinions through numerical ratings. Thus, supported by recent work on the field, we tackle the problem of fraud in such systems by designing a mechanism based on pairwise comparisons, coupled with an incentive policy attempting to foster the collection of majority opinions over individual experiences. As a result, we propose a new mechanism called iPWRM (incentive-based PWRM), where users are persuaded to reply honestly to pairwise queries based on opinion polls. The idea is: (1) to give a positive reward when all users agree in their reviews; (2) to give a positive reward when a user agrees the majority’s choice; and finally, (3) to give a low incentive—possibly null—when user’s review does not match the majority. Therefore, it is able (1) to overcome the bias introduced into reputation rankings by fraud reviews in ORSs, as well as (2) to mitigate potential biased problems derived from the use of numerical ratings. We exhaustively test the performance of the mechanism by using two different well-known existing datasets Flixster and HetRec2011—real world datasets on movie reviews, aiming to test the performance of the mechanism as well as the effectiveness and efficiency of iPWRM when fraud comes into play.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Notes

  1. For every match, a different set of users may be chosen. Moreover, note the size of this subset is very small compared to the size of the potential users that might be queried.

  2. It is important to remark that reward does not have to be necessarily money based, but it might be points, virtual money, or any other resource considered as valuable for users.

  3. http://www.cs.ubc.ca/~jamalim/datasets/.

  4. http://www.flixster.com.

  5. http://ir.ii.uam.es/hetrec2011.

  6. http://www.grouplens.org.

  7. http://www.movielens.org/.

  8. It is assumed users always reply a match query.

  9. Note that a detailed research on the performance of different configurations for PWRM set-up can be found in the works presented by Centeno et al. [4] and Hermoso et al. [14].

References

  1. Akoglu L, Chandy R, Faloutsos C (2013) Opinion fraud detection in online reviews by network effects. In: ICWSM

  2. Balakrishnan S, Chopra S (2010) Two of a kind or the ratings game? Adaptive pairwise preferences and latent factor models. In: ICDM, pp 725–730

  3. Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci 99(suppl 3):7280–7287

    Article  Google Scholar 

  4. Centeno R, Hermoso R, Fasli M (2014) On the inaccuracy of numerical ratings: dealing with biased opinions in social networks. Inf Syst Front 17:1–17

    Google Scholar 

  5. Chen K.-Y, Fine LR, Huberman BA (2001) Forecasting uncertain events with small groups. In: ACM EC, pp 58–64

  6. Dalvi N, Kumar R, Pang B (2013) Para ’normal’ activity: on the distribution of average ratings. In: ICWSM, pp 110–119

  7. Dellarocas C (2000) Mechanisms for coping with unfair ratings and discriminatory behavior in online reputation reporting systems. In: ICIS, pp 520–525

  8. De Meo P, Musial-Gabrys K, Rosaci D, Sarnè GML, Aroyo L (2017) Using centrality measures to predict helpfulness-based reputation in trust networks. ACM Trans Internet Technol 17:8:1–8:20. doi:10.1145/2981545

  9. Duan W, Gu B, Whinston AB (2008) Do online reviews matter? An empirical investigation of panel data. Decis Support Syst 45(4):1007–1016

    Article  Google Scholar 

  10. Fang H, Zhang J, Bao Y, Zhu Q (2013) Towards effective online review systems in the chinese context: a cross-cultural empirical study. Electron Commer Res Appl 12(3):208–220

    Article  Google Scholar 

  11. Farahbakhsh R, Ángel C, Crespi N (2015) Characterization of cross-posting activity for professional users across major osns. In: 2015 IEEE/ACM international conference on advances in social networks analysis and mining, pp 645–650

  12. Forsythe R, Rietz TA, Ross TW (1999) Wishes, expectations and actions: a survey on price formation in election stock markets. J Econ Behav Organ 39(1):83–110

    Article  Google Scholar 

  13. Golbeck J, Hansen D (2014) A method for computing political preference among twitter followers. Soc Netw 36:177–184 (Special Issue on Political Networks)

    Article  Google Scholar 

  14. Hermoso R, Centeno R, Fasli M (2013) From blurry numbers to clear preferences: a mechanism to extract reputation in social networks. Expert Syst Appl 41(5):2269–2285

    Article  Google Scholar 

  15. Hu N, Bose I, Koh NS, Liu L (2012) Manipulation of online reviews: an analysis of ratings, readability, and sentiments. Decis Support Syst 52(3):674–684

    Article  Google Scholar 

  16. Hu N, Liu L, Sambamurthy V (2011) Fraud detection in online consumer reviews. Decis Support Syst 50(3):614–626

    Article  Google Scholar 

  17. Huynh TD, Jennings NR, Shadbolt NR (2004) Fire: an integrated trust and reputation model for open multi-agent systems. In: Proceedings of the 16th European conference on artificial intelligence, ECAI’04. IOS Press, Amsterdam, pp 23–27

  18. Huynh TD, Jennings NR, Shadbolt NR (2006) An integrated trust and reputation model for open multi-agent systems. Auton Agents Multi Agent Syst 13(2):119–154

    Article  Google Scholar 

  19. Ismail R, Josang A (2002) The beta reputation system. In: BLED 2002 proceedings, p 41

  20. Jaramillo JJ, Srikant R (2010) A game theory based reputation mechanism to incentivize cooperation in wireless ad hoc networks. Ad Hoc Netw 8(4):416–429

    Article  Google Scholar 

  21. Jindal N, Liu B (2008) Opinion spam and analysis. In: Proceedings of the 2008 international conference on web search and data mining. ACM, pp 219–230

  22. Jøsang A, Ismail R (2002) The beta reputation system. In: Bled eConference

  23. Jøsang A, Ismail R, Boyd C (2007) A survey of trust and reputation systems for online service provision. Decis Support Syst 43(2):618–644

    Article  Google Scholar 

  24. Jurca R, Faltings B (2008) Incentives for expressing opinions in online polls. In: ACM EC, pp 119–128

  25. Kendall MG (1938) A new measure of rank correlation. Biometrika 30(1/2):81–93

    Article  MATH  Google Scholar 

  26. Khosravifar B, Bentahar J, Gomrokchi M, Alam R (2012) CRM: an efficient trust and reputation model for agent computing. Knowl Based Syst 30:1–16

    Article  Google Scholar 

  27. Koh NS, Hu N, Clemons EK (2010) Do online reviews reflect a products true perceived quality? An investigation of online movie reviews across cultures. Electron Commer Res Appl 9(5):374–385

    Article  Google Scholar 

  28. Kost A (2012) Woman paid to post five-star google feedback. http://www.thedenverchannel.com/news/woman-paid-to-post-five-star-google-feedback

  29. Koutsoupias E, Papadimitriou C (1999) Worst-case equilibria. In: Proceedings of the 16th annual conference on theoretical aspects of computer science, STACS’99. Springer, Berlin, pp 404–413

  30. Lerman K, Hogg T (2014) Leveraging position bias to improve peer recommendation. PLOS ONE 9(6):1–8

    Article  Google Scholar 

  31. Lewis-Beck MS, Skalaban A (1989) Citizen forecasting: can voters see into the future? Br J Polit Sci 19:146–153

    Article  Google Scholar 

  32. Lim E-P, Nguyen V-A, Jindal N, Liu B, Lauw HW (2010) Detecting product review spammers using rating behaviors. In: Proceedings of the 19th ACM international conference on Information and knowledge management. ACM, pp 939–948

  33. Moon S, Bergey PK, Iacobucci D (2010) Dynamic effects among movie ratings, movie revenues, and viewer satisfaction. J Mark 74(1):108–121

    Article  Google Scholar 

  34. Negahban S, Oh S, Shah D (2012) Iterative ranking from pair-wise comparisons. In: NIPS, pp 2483–2491

  35. Pinyol I, Sabater-Mir J (2013) Computational trust and reputation models for open multi-agent systems: a review. Artif Intell Rev 40(1):1–25

    Article  Google Scholar 

  36. Prelec D (2004) A bayesian truth serum for subjective data. Science 306(5695):462–466

    Article  Google Scholar 

  37. Ramchurn SD, Huynh D, Jennings NR (2004) Trust in multi-agent systems. Knowl Eng Rev 19(1):1–25

    Article  Google Scholar 

  38. Resnick P, Kuwabara K, Zeckhauser R, Friedman E (2000) Reputation systems. Commun ACM 43(12):45–48

    Article  Google Scholar 

  39. Sabater J, Sierra C (2001) Regret: reputation in gregarious societies. In: Proceedings of the fifth international conference on autonomous agents, AGENTS ’01. ACM, New York, pp 194–195. doi:10.1145/375735.376110

  40. Sparling EI, Sen S (2011) Rating: how difficult is it?. In: RecSys, pp 149–156

  41. Streitfeld D (2011) In a race to out-rave, 5-star web reviews go for $5’. http://www.nytimes.com/2011/08/20/technology/finding-fake-reviews-online.html

  42. Teacy WL, Luck M, Rogers A, Jennings NR (2012) An efficient and versatile approach to trust and reputation using hierarchical bayesian modelling. Artif Intell 193:149–185

    MathSciNet  Article  MATH  Google Scholar 

  43. Teacy WTL, Patel J, Jennings NR, Luck M (2006) Travos: Trust and reputation in the context of inaccurate information sources. Auton Agents Multi Agent Syst 12(2):183–198

    Article  Google Scholar 

  44. Vu T, Altman A, Shoham Y (2009) On the complexity of schedule control problems for knockout tournaments. In: Proceedings of AAMAS’09, vol  1 of AAMAS ’09. IFAAMAS, Richland, pp 225–232

  45. Vu T, Shoham Y (2011) Fair seeding in knockout tournaments. ACM Trans Intell Syst Technol 3(1):9:1–9:17

    Article  Google Scholar 

  46. Wang Y, Vassileva J (2007) Toward trust and reputation based web service selection: a survey. Int Trans Syst Sci Appl 3(2):118–132

    Google Scholar 

  47. Witkowski J, Bachrach Y, Key P, Parkes DC (2013) Dwelling on the negative: incentivizing effort in peer prediction. In: HCOMP

  48. Wooldridge M (2009) An introduction to multiagent systems, 2nd edn. Wiley Publishing, London

    Google Scholar 

  49. Xie S, Wang G, Lin S, Yu PS (2012) Review spam detection via temporal pattern discovery. In: KDD, pp 823–831

  50. Yan Q, Wang Q, Liu X (2014) Research on the interactive effects of online scores. Electron Commer Res Appl 13(6):402–408

    Article  Google Scholar 

  51. Yu B, Singh MP (2002) An evidential model of distributed reputation management. In AAMAS, pp 294–301

Download references

Acknowledgements

We thank anonymous reviewers for their very useful comments and suggestions. The work was supported by eMadrid project S2013-ICE-2715, Spanish Ministry of Economy and Competitiveness (TIN2012-36586-C03-02-iHAS) and by the Autonomous Region of Madrid (P2013/ICE-3019-MOSI-AGIL-CM, co-funded by EU-FSE and FEDER”).

Author information

Authors and Affiliations

  1. Department of Computer Languages and Systems, UNED, Madrid, Spain

    Roberto Centeno

  2. Department of Informatics and Systems Engineering, University of Zaragoza, Zaragoza, Spain

    Ramón Hermoso

Authors
  1. Roberto Centeno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramón Hermoso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto Centeno.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Centeno, R., Hermoso, R. Estimating global opinions by keeping users from fraud in online review systems. Knowl Inf Syst 55, 467–491 (2018). https://doi.org/10.1007/s10115-017-1089-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-017-1089-2

Keywords

  • Online review systems
  • Fraud
  • Opinions
  • Reputation
  • Incentives