Skip to main content
SpringerLink
Log in

The Recipe for the Perfect Review?

An Investigation into the Determinants of Review Helpfulness

  • Research Paper
  • Published:
Business & Information Systems Engineering Aims and scope Submit manuscript

Abstract

Online product reviews, originally intended to reduce consumers’ pre-purchase search and evaluation costs, have become so numerous that they are now themselves a source for information overload. To help consumers find high-quality reviews faster, review rankings based on consumers’ evaluations of their helpfulness were introduced. But many reviews are never evaluated and never ranked. Moreover, current helpfulness-based systems provide little or no advice to reviewers on how to write more helpful reviews. Average review quality and consumer search costs could be much improved if these issues were solved. This requires identifying the determinants of review helpfulness, which we carry out based on an adaption of Wang and Strong’s well-known data quality framework. Our empirical analysis shows that review helpfulness is influenced not only by single-review features but also by contextual factors expressing review value relative to all available reviews. Reviews for experiential goods differ systematically from reviews for utilitarian goods. Our findings, based on 27,104 reviews from Amazon.com across six product categories, form the basis for estimating preliminary helpfulness scores for unrated reviews and for developing interactive, personalized review writing support tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Amazon-like review systems require reviewers to give an overall star rating and a free-format review (minimum length 20 words), but not to rate products on mandatory standardized criteria (e.g., TripAdvisor.com).

  2. Two studies on review helpfulness (Jin and Liu 2010; Chen and Tseng 2011) used parts of this framework to derive review quality measures.

  3. The original framework was developed in a business/enterprise database management context.

  4. The product features were collected from the manufacturers’ homepages.

  5. We re-ran our regressions with other readability indices (Flesch-Kincaid Readability Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, Automated Readability Index and Coleman-Liau Index) and found virtually no differences. We chose to present the model with SMOG because it produced the lowest values for the Akaike Information Criterion.

  6. The top 10 most (un)helpful adjectives and adverbs for each product category are listed in Online-Appendix D.

References

  • Benamara F, Cesarano C, Picariello A, Reforgiato D, Subrahmanian VS (2007) Sentiment analysis: adjectives and adverbs are better than adjectives alone. In: Proceedings of the 1st international AAAI conference on weblogs and social media, Boulder, CO

    Google Scholar 

  • Berger J, Sorensen AT, Rasmussen SJ (2010) Positive effects of negative publicity: when negative reviews increase sales. Marketing Sci 29(5):815–827

    Article  Google Scholar 

  • Bone PF (1995) Word-of-mouth effects on short-term and long-term product judgments: interpersonal buyer behavior in marketing. J Bus Res 32(3):213–223

    Article  Google Scholar 

  • Burton J, Khammash M (2010) Why do people read reviews posted on consumer-opinion portals? J Mark Manag 26(3):230–255

    Article  Google Scholar 

  • Cacioppo JT, Petty RE (1984) The elaboration likelihood model of persuasion. Adv Consum Res 11:673–675

    Google Scholar 

  • Chen CC, Tseng Y (2011) Quality evaluation of product reviews using an information quality framework. Decis Support Syst 50(4):755–768

    Article  Google Scholar 

  • Chen P, Dhanasobhon S, Smith MD (2008) All reviews are not created equal: the disaggregate impact of reviews and reviewers at Amazon.com. SSRN working paper. http://ssrn.com/abstract=918083. Accessed 2013-02-25

  • Connors L, Mudambi SM, Schuff D (2011) Is it the review or the reviewer? A multi-method approach to determine the antecedents of online review helpfulness. In: Proceedings of the 44th Hawaii international conference on systems science (HICSS), Hawaii, USA

    Google Scholar 

  • Danescu-Niculescu-Mizil C, Kossinets G, Kleinberg JM, Lee L (2009) How opinions are received by online communities: a case study on Amazon.com helpfulness votes. In: Quemada J, León G, Maarek Y, Nejdl W (eds) Proceedings of the 18th international conference on world wide web. ACM, New York

    Google Scholar 

  • Dellarocas C (2003) The digitization of word of mouth: promise and challenges of online feedback mechanisms. Manag Sci 49(10):1407–1424

    Article  Google Scholar 

  • Dellarocas C, Gao G, Narayan R, (2010) Are consumers more likely to contribute online reviews for hit or Niche products? J Man Inf Sys 27(2):127–157

    Article  Google Scholar 

  • Eagly AH (1974) Comprehensibility of persuasive arguments as a determinant of opinion change. J Pers Soc Psychol 29(6):758–773

    Article  Google Scholar 

  • Eagly AH, Chaiken S (1984) Cognitive theories of persuasion. In: Berkowitz L (ed) Advances in experimental social psychology. Academic Press, San Diego, pp 267–359

    Google Scholar 

  • Feldman JM, Lynch JG (1988) Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior. J Appl Psychol 73(3):421–435

    Article  Google Scholar 

  • Folkes VS (1988) Recent attribution research in consumer behavior: a review and new directions. J Consum Res 14(4):548–565

    Article  Google Scholar 

  • Ford GT, Smith DB, Swasy JL (1990) Consumer skepticism of advertising claims: testing hypotheses from economics of information. J Consum Res 16(4):433–441

    Article  Google Scholar 

  • Forman C, Ghose A, Wiesenfeld B (2008) Examining the relationship between reviews and sales: the role of reviewer identity disclosure in electronic markets. Inf Syst Res 19(3):291–313

    Article  Google Scholar 

  • Ghose A, Ipeirotis PG (2011) Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Trans Knowl Data Eng 23(10):1498–1512

    Article  Google Scholar 

  • Greene WH (2012) Econometric analysis, 7th edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Hao Y, Li Y, Zou P (2009) Why some online product reviews have no usefulness rating? In: Proceedings of the Pacific Asia conference on information systems (PACIS 2009), Hyderabad, India, Paper 100

    Google Scholar 

  • Herr PM, Kardes FR, Kim J (1991) Effects of word-of-mouth and product-attribute information on persuasion: an accessibility-diagnosticity perspective. J Consum Res 17(4):454–462

    Article  Google Scholar 

  • Jain SP, Posavac SS (2001) Prepurchase attribute verifiability, source credibility, and persuasion. J Consum Psychol 11(3):169–180

    Article  Google Scholar 

  • Jin J, Liu Y (2010) How to interpret the helpfulness of online product reviews: bridging the needs between customers and designers. In: Cortizo JC, Carrero FM, Cantador I, Troyano JA, Rosso P (eds) Proceedings of the 2nd international workshop on search and mining user-generated contents. ACM, New York, pp 87–94

    Chapter  Google Scholar 

  • Kim S, Pantel P, Chklovski T, Pennacchiotti M (2006) Automatically assessing review helpfulness. In: Proceedings of the 2006 conference on empirical methods in natural language processing. Association for computational linguistics. Sydney, Australia, pp 423–430

    Google Scholar 

  • Klare GR (2000) The measurement of readability: useful information for communicators. ACM J Comput Doc 24(3):11–25

    Google Scholar 

  • Korfiatis N, Rodríguez D, Sicilia M (2008) The impact of readability on the usefulness of online product reviews: a case study on an online bookstore. In: Lytras MD, Carroll JM, Damiani E, Tennyson RD (eds) Emerging technologies and information systems for the knowledge society. Springer, Heidelberg, pp 423–432

    Chapter  Google Scholar 

  • Lahiri S, Mitra P, Lu X (2011) Informationality judgement at sentence level and experiments with formality score. In: Proceedings of the 12th international conference on computational linguistics and intelligent text processing, Tokio

    Google Scholar 

  • Li MX, Huang L, Tan CH, Wei KK (2011) Assessing the helpfulness of online product review: a progressive experimental approach. In: Seddon PB, Gregor S (eds) Proceedings of the Pacific Asia conference on information systems (PACIS 2011), Brisbane, Australia, Paper 111

    Google Scholar 

  • Li X, Hitt LM (2008) Self-selection and information role of online product reviews. Inf Syst Res 19(4):456–474

    Article  Google Scholar 

  • Liu J, Cao Y, Lin C, Huang Y, Zhou M (2007) Low-quality product review detection in opinion summarization. In: Proceedings of the joint conference on empirical methods in natural language processing and computational natural language learning (EMN-CoNLL), pp 334–342

    Google Scholar 

  • Liu Y, Huang X, An A, Yu X (2008) HelpMeter: a nonlinear model for predicting the helpfulness of online reviews. In: Proceedings of the 2008 IEEE/WIC/ACM international conference on web intelligence and intelligent agent technology, Sydney, Australia, pp 793–796

    Chapter  Google Scholar 

  • McLaughlin HG (1969) SMOG grading – a new readability formula. J Read 12(8):639–646

    Google Scholar 

  • Mudambi SM, Schuff D (2010) What makes a helpful online review? A study of customer reviews on Amazon.com. MIS Q 34(1):185–200

    Google Scholar 

  • Nelson P (1970) Information and consumer behavior. J Polit Econ 78(20):311–329

    Article  Google Scholar 

  • Netzer O, Srinivasan V (2011) Adaptive self-explication of multiattribute preferences. J Mark Res 48(1):140–156

    Google Scholar 

  • Otterbacher J (2008) Managing information in online product review communities: two approaches. In: Golden W, Acton T, Conboy K, van der Heijden H, Tuunainen VK (eds) Proceedings of the 16th European conference on information systems, pp 706–717

    Google Scholar 

  • Pan Y, Zhang JQ (2011) Born unequal: a study of the helpfulness of user-generated product reviews. J Retail 87(4):598–612

    Article  Google Scholar 

  • Park DY, Lee J, Han I (2007) The effect of on-line consumer reviews on consumer purchasing intention: the moderating role of involvement. Int J Electron Commer 11(4):125–148

    Article  Google Scholar 

  • Schindler RM, Bickart B (2012) Perceived helpfulness of online consumer reviews: the role of message content and style. J Consum Behav 11:234–243

    Article  Google Scholar 

  • Schlosser AE (2011) Can including pros and cons increase the helpfulness and persuasiveness of online reviews? The interactive effects of ratings and arguments. J Consum Psychol 21(3):226–239

    Google Scholar 

  • Scholz SW, Meissner M, Decker R (2010) Measuring consumer preferences for complex products: a compositional approach based on paired comparisons. J Mark Res 47(4):685–698

    Article  Google Scholar 

  • Schwenk CR (1986) Information, cognitive biases, and commitment to a course of action. Acad Manag Rev 11(2):298–310

    Google Scholar 

  • Sen S, Lerman D (2007) Why are you telling me this? An examination into negative consumer reviews on the web. J Interact Mark 21(4):76–94

    Article  Google Scholar 

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423

    Google Scholar 

  • Toutanova K, Manning CD (2000) Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proceedings of the joint SIGDAT conference on empirical methods in natural language processing and very large corpora, Hong Kong, China

    Google Scholar 

  • Tversky A, Kahneman D (1974) Judgment under uncertainty: heuristics and biases. Science 185(4157):1124–1131

    Article  Google Scholar 

  • Wang B, Zhu W, Chen L (2011) Improving the Amazon review system by exploiting the credibility and time-decay of public reviews. Informatica 35(4):463–472

    Google Scholar 

  • Wang RW, Strong DM (1996) Beyond accuracy: what data quality means to data consumers. J Manag Inf Syst 12(4):5–33

    Google Scholar 

  • Weathers D, Sharma S, Wood SL (2007) Effects of online communication practices on consumer perceptions of performance uncertainty for search and experience goods. J Retail 83(4):393–401

    Article  Google Scholar 

  • Wright DB, London K (2009) Modern regression techniques using R. Sage, London

    Google Scholar 

  • Wu P, van der Heijden H, Korfiatis N (2011) The influences of negativity and review quality on the helpfulness of online reviews. In: Galletta DF, Liang T (eds) Proceedings of the international conference on information systems, Shanghai, China

    Google Scholar 

  • Xia L, Bechwati NN (2011) Word of mouse: the role of cognitive personalization in online consumer reviews. J Interactive Advertising 9(1):3–13

    Google Scholar 

  • Xia R, Zong C, Li S (2011) Ensemble of feature sets and classification algorithms for sentiment classification. Inf Sci 181(6):1138–1152

    Article  Google Scholar 

  • Zhang JQ, Craciun G, Shin D (2010) When does electronic word-of-mouth matter? A study of consumer product reviews. J Bus Res 63(12):1336–1341

    Article  Google Scholar 

  • Zhang R, Tran T (2010) Helpful or unhelpful: a linear approach for ranking product reviews. Electron Commer Res 11(3):220–230

    Google Scholar 

  • Zhang R, Tran T (2011) An information gain-based approach for recommending useful product reviews. Knowl Inf Syst 26(3):419–434

    Article  Google Scholar 

  • Zhang Z, Varadarajan B (2006) Utility scoring of product reviews. In: Yu PS, Tsotras V, Fox EA, Liu B (eds) Proceedings of the 15th ACM international conference on information and knowledge management, Arlington, VA. ACM, New York, p 51

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Juniorprofessur für E-Commerce, University Passau, Innstr. 43, 94032, Passau, Germany

    Michael Scholz

  2. Chair of Business Computing II, University Passau, Innstr. 43, 94032, Passau, Germany

    Verena Dorner

Authors
  1. Michael Scholz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Verena Dorner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Verena Dorner.

Additional information

Accepted after two revisions by Prof. Dr. Spann.

This article is also available in German in print and via http://www.wirtschaftsinformatik.de: Scholz M, Dorner V (2013) Das Rezept für die perfekte Rezension? Einflussfaktoren auf die Nützlichkeit von Online-Kundenrezensionen. WIRTSCHAFTSINFORMATIK. doi: 10.1007/s11576-013-0358-2.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(PDF 50 kB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Scholz, M., Dorner, V. The Recipe for the Perfect Review?. Bus Inf Syst Eng 5, 141–151 (2013). https://doi.org/10.1007/s12599-013-0259-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12599-013-0259-3

Keywords

Search

Navigation