Abstract
In this study, we aim to clarify the determinants of online review helpfulness concerning review depth, extremity and timeliness. Based on a meta-analysis, we examine the effects of important characteristics of reviews employing 53 empirical studies yielding 191 effect sizes. Findings reveal that review depth has a greater impact on helpfulness than review extremity and timeliness with the exception of its sub-metric of review volume, which exerts the negative influence on review helpfulness. Specifically, readability is the most important factor in evaluating review helpfulness. Furthermore, we discuss important moderators of the relationships and find interesting insights regarding website and culture background. In accordance with the results, we propose several implications for researchers and E-business firms. Our study provides a much needed quantitative synthesis of this burgeoning stream of research.
Similar content being viewed by others
Notes
We convert t value to the correlation coefficient effect size by using the formula suggested by Rosenthal [46]: \({\text{r}} = {\raise0.7ex\hbox{${\text{t}}$} \!\mathord{\left/ {\vphantom {{\text{t}} {\sqrt {\left( {{\text{t}}^{2} + {\text{d}}.{\text{f}}.} \right)} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sqrt {\left( {{\text{t}}^{2} + {\text{d}}.{\text{f}}.} \right)} }$}}\) where \({\text{t}}\) is the t-value associated with the regression parameter that captures the effect and \({\text{d}}. {\text{f}}.\) is the degree of freedom of the reported regression model; We convert \(\beta\) to coefficient effect size by using the formula suggested by Peterson and Brown [47]: \(r = 0.98\beta + 0.05\lambda\), where \(\lambda\) is a variable that equals 1 when \(\beta\) is non-negative and 0 when \(\beta\) is negative.
We use the following formulae for Fisher’s Z: (1) transformation: \(z_{r} = 0.5{ \ln }\left( {{\raise0.7ex\hbox{${\left( {1 + r} \right)}$} \!\mathord{\left/ {\vphantom {{\left( {1 + r} \right)} {1 - r}}}\right.\kern-0pt} \!\lower0.7ex\hbox{${1 - r}$}}} \right)\), (2) Weighted average: \(\overline{{z_{r} }} = {\raise0.7ex\hbox{${\sum \left( {n_{i} - 3} \right)*z_{r} }$} \!\mathord{\left/ {\vphantom {{\sum \left( {n_{i} - 3} \right)*z_{r} } {\sum \left( {n_{i} - 3} \right)}}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sum \left( {n_{i} - 3} \right)}$}}\) and (3) back-transformation to correlation units:
$$\bar{r} = {\raise0.7ex\hbox{${\left( {e^{{2\overline{{z_{r} }} }} - 1} \right)}$} \!\mathord{\left/ {\vphantom {{\left( {e^{{2\overline{{z_{r} }} }} - 1} \right)} {\left( {e^{{2\overline{{z_{r} }} }} + 1} \right)}}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\left( {e^{{2\overline{{z_{r} }} }} + 1} \right)}$}}\;[49].$$We calculate the 95% confidence interval as: lower \(CI = \overline{{z_{r} }} - {\raise0.7ex\hbox{${1.96}$} \!\mathord{\left/ {\vphantom {{1.96} {\sqrt {\sum \left( {n_{i} - 3} \right)} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sqrt {\sum \left( {n_{i} - 3} \right)} }$}}\), upper \(CI = \overline{{z_{r} }} + {\raise0.7ex\hbox{${1.96}$} \!\mathord{\left/ {\vphantom {{1.96} {\sqrt {\sum \left( {n_{i} - 3} \right)} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sqrt {\sum \left( {n_{i} - 3} \right)} }$}}\); the variance of effect size as: \(S_{r}^{2} = {\raise0.7ex\hbox{${\sum n_{i} \left( {r_{i} - \bar{r}} \right)^{2} }$} \!\mathord{\left/ {\vphantom {{\sum n_{i} \left( {r_{i} - \bar{r}} \right)^{2} } {\sum n_{i} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sum n_{i} }$}}\) and variation caused by sampling error as: \(S_{e}^{2} = {\raise0.7ex\hbox{${\sum n_{i} \left( {1 - \bar{r}} \right)^{2} }$} \!\mathord{\left/ {\vphantom {{\sum n_{i} \left( {1 - \bar{r}} \right)^{2} } {\sum n_{i} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${\sum n_{i} }$}}\); Q-value is calculated as: \(Q = \sum \left( {n_{i} - 3} \right)*\left( {z_{r} - \overline{{z_{r} }} } \right)^{2}\) and the fail safe N is as: \(N = k*\left( {{\raise0.7ex\hbox{${\bar{r}}$} \!\mathord{\left/ {\vphantom {{\bar{r}} {r_{c} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${r_{c} }$}} - 1} \right)\), where \(r_{c}\) is the “just significant” level or critical effect size which usually use 0.01.
References
Dellarocas, C. (2003). The digitization of word of mouth: Promise and challenges of online feedback mechanisms. Management Science, 49(10), 1407–1424.
Malik, M., & Hussain, A. (2018). An analysis of review content and reviewer variables that contribute to review helpfulness. Information Processing and Management, 54(1), 88–104.
Mudambi, S. M., & Schuff, D. (2010). What makes a helpful online review? A study of customer reviews on amazon.com. MIS Quarterly, 34(1), 185–200.
Ghose, A., & Ipeirotis, P. G. (2011). Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering, 23(10), 1498–1512.
Yin, G., Wei, L., Xu, W., & Chen, M. (2014). Exploring heuristic cues for consumer perceptions of online reviews helpfulness: The case of yelp.com. In Pacific Asia conference on information systems.
Farley, J. U., Lehmann, D. R., & Sawyer, A. (1995). Empirical marketing generalizations using meta-analysis. Marketing Science, 14(3), G36–G46.
Floyd, K., Freling, R., Alhoqail, S., Cho, H. Y., & Freling, T. (2014). How online product reviews affect retail sales: A meta-analysis. Journal of Retailing, 90(2), 217–232.
You, Y., Vadakkepatt, G., & Joshi, A. M. (2015). A meta-analysis of electronic word-of-mouth elasticity. Journal of Marketing, 79(2), 19–39.
Rosario, A. B., Sotgiu, F., Valck, K. D., & Bijmolt, T. H. A. (2016). The effect of electronic word of mouth on sales: A meta-analytic review of platform, product, and metric factors. Journal of Marketing Research, 53(3), 297–318.
Purnawirawan, N., Eisend, M., Pelsmacker, P. D., & Dens, N. (2015). A meta-analytic investigation of the role of valence in online reviews. Journal of Interactive Marketing, 31, 17–27.
Pan, Y., & Zhang, J. Q. (2011). Born unequal: A study of the helpfulness of user-generated product reviews. Journal of Retailing, 87(4), 598–612.
Ghose, A., & Ipeirotis, P. G. (2010). Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Transactions on Knowledge and Data Engineering, 23(10), 1498–1512.
Ogut, H., & Tas, B. (2012). The influence of internet customer reviews on the online sales and prices in hotel industry. The Service Industry Journal, 32(2), 197–214.
Chu, W., & Roh, M. (2014). Exploring the role of preference heterogeneity and causal attribution in online ratings dynamics. Asia Marketing Journal, 15(4), 61–101.
Duan, W., Gu, B., & Whinston, A. (2008). Do online reviews matter?—An empirical investigation of panel data. Decision Support Systems, 45(4), 1007–1016.
Hu, N., Liu, L., & Zhang, J. (2008). Do online reviews affect product sales? The role of reviewer characteristics and temporal effects. Information Technology Management, 9(3), 201–214.
Cui, G., Lui, H. K., & Guo, X. (2012). The effect of online consumer reviews on new product sales[M]. International Journal of Electronic Commerce, 17(1), 39–57.
Cao, Q., Duan, W., & Gan, Q. (2011). Exploring determinants of voting for the “helpfulness” of online user reviews: A text mining approach. Decision Support Systems, 50(2), 511–521.
Korfiatis, N., García, B. E., & Sánchez, A. S. (2012). Evaluating content quality and helpfulness of online product reviews: The interplay of review helpfulness versus review content. Electronic Commerce Research and Applications, 11(3), 205–217.
Baek, H., Ahn, J., & Choi, Y. (2012). Helpfulness of online consumer reviews: Readers’ objectives and review cues. International Journal of Electronic Commerce, 17(2), 99–126.
Lee, S., & Choeh, J. Y. (2014). Predicting the helpfulness of online reviews using multilayer perceptron neural networks. Expert Systems with Applications, 41(6), 3041–3046.
Huang, A. H., Chen, K., Yen, D. C., & Tran, T. P. (2015). A study of factors that contribute to online review helpfulness. Computers in Human Behavior, 48(C), 17–27.
Fu, D., Hong, Y., Wang, K., & Fan, W. (2017). Effects of membership tier on user content generation behaviors: Evidence from online reviews. Electronic Commerce Research, 7, 1–27.
Singh, J., Irani, S., & Rana, N. (2016). Predicting the “helpfulness” of online consumer reviews. Journal of Business Research, 70, 346–355.
Otterbacher, J. (2009). ‘Helpfulness’ in online communities: a measure of message quality. In Sigchi conference on human factors in computing systems (pp. 955–964).
Baek, H., Lee, S., Oh, S., & Ahn, J. (2015). Normative social influence and online review helpfulness: Polynomial modeling and response surface analysis. Journal of Electronic Commerce Research, 16(4), 290–306.
Liu, Z., & Park, S. (2015). What makes a useful online review? Implication for travel product websites. Tourism Management, 47(47), 140–151.
Ngoye, T. L., & Sinha, A. P. (2014). The influence of reviewer engagement characteristics on online review helpfulness: A text regression model. Decision Support Systems, 61(4), 47–58.
Xiang, Z., Du, Q., Ma, Y., & Fan, W. (2017). A comparative analysis of major online review platforms: Implications for social media analytics in hospitality and tourism. Tourism Management, 58, 51–65.
Wu, J. (2017). Review popularity and review helpfulness: A model for user review effectiveness. Decision Support Systems, 97, 92–103.
Guo, B., & Zhou, S. (2017). What makes population perception of review helpfulness: An information processing perspective. Electronic Commerce Research, 17(4), 1–24.
Chen, Z., & Lurie, N. (2013). Temporal contiguity and negativity bias in the impact of online word of mouth. Journal of Marketing Research, 50(4), 463–476.
Kohli, R., Devaraj, S., & Mahmood, M. A. (2004). Understanding determinants of online consumer satisfaction: A decision process perspective. Journal of Management Information Systems, 21(1), 115–135.
Qazi, A., Syed, K., Raj, R. G., & Cambria, E. (2016). A concept-level approach to the analysis of online review helpfulness. Computers in Human Behavior, 58(C), 75–81.
Racherla, P., & Friske, W. (2012). Perceived usefulness of online consumer reviews: An exploratory investigation across three services categories. Electronic Commerce Research and Applications, 11(6), 548–559.
Schindler, R. M., & Bickart, B. (2012). Perceived helpfulness of online consumer reviews: The role of message content and style. Journal of Consumer Behavior, 11(3), 234–342.
Kuan, K., & Hui, K. (2015). What makes a review voted? An empirical investigation of review voting in online review systems. Journal of the Association for Information Systems, 16(1), 48–71.
Anderson, W. E., & Salisbury, C. L. (2003). The formation of market-level expectations and its covariates. Journal of Consumer Research, 30(1), 115–124.
Khare, A., Lauren, I. L., & Anthony, K. A. (2011). The assimilative and contrastive effects of word-of-mouth volume: An experimental examination of online consumer ratings. Journal of Retailing, 87(1), 111–126.
Salganik, M. J., Peter, S. D., & Duncan, J. W. (2006). Experimental study of inequality and unpredictability in an artificial cultural market. Science, 311(5762), 854–856.
Forman, C., Ghose, A., & Wiesenfeld, B. (2008). Examining the relationship between reviews and sales: The role of reviewer identity disclosure in electronic markets. Information Systems Research, 19(3), 291–313.
Zhou, S., & Guo, B. (2015). The interactive effect of review rating and text sentiment on review helpfulness. In E-Commerce and web technologies (pp. 100–111).
Zhu, L., Yin, G., & He, W. (2014). Is this opinion leader’s review useful? Peripheral cues for online review helpfulness. Journal of Electronic Commerce Research, 15(4), 267–280.
Li, J., & Zhan, L. (2011). Online persuasion: How the written word drives WOM—Evidence from consumer-generated product reviews. Journal of Advertising Research, 51(1), 239–257.
Chua, A. Y. K., & Banerjee, S. (2016). Helpfulness of user-generated reviews as a function of review sentiment, product type and information quality. Computers in Human Behavior, 54(C), 547–554.
Kim, S. M., Pantel, P., Chklovski, T., & Pennacchiotti, M. (2006). Automatically assessing review helpfulness. In Conference on empirical methods in natural language processing (pp. 423–430). Association for Computational Linguistics.
Wu, P. F., Hans, V. D. H., & Korfiatis, N. (2011). The influences of negativity and review quality on the helpfulness of online reviews. In International conference on information systems. Shanghai, China.
Yin, D., Bond, S. D., & Zhang, H. (2014). Anxious or angry? Effects of discrete emotions on the perceived helpfulness of online reviews. MIS Quarterly, 38(2), 539–560.
Chua, A. Y. K., & Banerjee, S. (2015). Understanding review helpfulness as a function of reviewer reputation, review rating, and review depth. Journal of the Association for Information Science and Technology, 66(2), 354–362.
Sangwon, P., & Nicolau, J. L. (2015). Asymmetric effects of online consumer reviews. Annals of Tourism Research, 50, 67–83.
Wan, Y. (2015). The Matthew effect in social commerce: The case of online review helpfulness. Electronic Markets, 25(4), 313–324.
Salehan, M., & Dan, J. K. (2016). Predicting the performance of online consumer reviews. Decision Support Systems, 81(C), 30–40.
Koh, N. S., Hu, N., & Clemons, E. K. (2010). Do online reviews reflect a product’s true perceived quality? An investigation of online movie reviews across cultures. Electronic Commerce Research and Applications, 9(5), 374–385.
Mcginnies, E., & Ward, D. C. (1980). Better liked than right: Trustworthiness and expertise as factors in credibility. Personality and Social Psychology Bulletin, 6(3), 467–472.
Grewal, D., Gotlieb, J., & Marmorstein, H. (1994). The moderating effects of message framing and source credibility on the price-perceived risk relationship. Journal of Consumer Research, 21(1), 145–153.
Schweidel, D., & Moe, W. (2014). Listening in on social media: A joint model of sentiment and venue format choice. Journal of Marketing Research, 51(4), 387–402.
Johnson, D. W., Maruyama, G., Johnson, R., Nelson, D., & Skon, L. (1981). Effects of cooperative, competitive and individualistic goal structures on achievement: A meta-analysis. Psychological Bulletin, 89(1), 47–62.
Assmus, G., Farley, J. U., & Lehmann, D. R. (1984). How advertising affects sales: Meta-analysis of econometric results. Journal of Marketing Research, 21(1), 65–74.
Lee, J. (2013). What makes people read an online review? The relative effects of posting time and helpfulness on review readership. Cyberpsychology Behavior & Social Networking, 16(7), 529.
Völckner, F., & Hofmann, J. (2007). The price-perceived quality relationship: A meta-analytic review and assessment of its determinants. Marketing Letters, 18(3), 181–196.
Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231–244). New York: Russell Sage Foundation.
Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90(1), 175–181.
Hunter, J. E., & Schmidt, F. L. (2006). Methods of meta-analysis: Correcting error and bias in research findings. Evaluation & Program Planning, 29(3), 236–237.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Beverly Hills: Sage.
Rosenthal, R. (1979). The ‘file drawer problem’ and tolerance for null results. Psychological Bulletin, 86, 638–641.
Tellis, G. J. (1988). The price elasticity of selective demand: A meta-analysis of econometric models of sales. Journal of Marketing Research, 25(4), 331–341.
Singer, J. D., & Willet, J. B. (2003). Applied longitudinal data analysis. Berlin: Springer.
Bijmolt, T. H. A., & Pieters, R. G. M. (2001). Meta-analysis in marketing when studies contain multiple measurements. Marketing Letters, 12(2), 157–169.
Hair, J. F., Anderson, R. E., Tatham, R. L., & Black W. C. (1998). Multivariate data analysis (5th ed.). All Publications.
Huang, H. (2005). A cross-cultural test of the spiral of silence. International Journal of Public Opinion Research, 17(3), 324–345.
Park, D. H., & Kim, S. (2009). The effects of consumer knowledge on message processing of electronic word-of-mouth via online consumer reviews. Electronic Commerce Research and Applications, 7(4), 399–410.
Vessey, I., & Galletta, D. (1991). Cognitive fit: An empirical study of information acquisition. Information Systems Research, 2(1), 63–84.
Jackson, T. W., & Farzaneh, P. (2012). Theory-based model of factors affecting information overload. International Journal of Information Management, 32(6), 523–532.
Schultz, C., Schreyoegg, J., & Von, R. C. (2013). The moderating role of internal and external resources on the performance effect of multitasking: Evidence from the R&D performance of surgeons. Research Policy, 42(8), 1356–1365.
Jiang, Z., & Benbasat, I. (2007). Investigating the influence of the functional mechanisms of online product presentations. Information Systems Research, 18(4), 221–244.
Cohen, J. (1988). Statistical power analysis for the behavioral science. Journal of the American Statistical Association, 84(363), 19–74.
Huang, H. A., & Yen, C. D. (2013). Predicting the helpfulness of online reviews—A replication. International Journal of Human-computer Interaction, 29(2), 129–138.
Zhang, Z. (2008). Weighing stars: Aggregating online product reviews for intelligent E-commerce applications. IEEE Intelligent Systems, 23(5), 42–49.
Willemsen, L. M., Neijens, P. C., Bronner, F., & Ridder, J. (2011). “Highly Recommended!” The content characteristics and perceived usefulness of online consumer reviews. Journal of Computer-Mediated Communication, 17(1), 19–38.
Siering, M., Muntermann, J., & Rajagopalan, B. (2018). Explaining and predicting online review helpfulness: The role of content and reviewer-related signals. Decision Support Systems, 108, 1–12.
Zhang, J. Q., Craciun, G., & Shin, D. (2010). When does electronic word-of-mouth matter? A study of consumer product reviews. Journal of Business Research, 63(12), 1336–1341.
Einar, B., Havro, L. J., & Moen, O. (2015). An empirical investigation of self-selection bias and factors influencing review helpfulness. International Journal of Business & Management, 10(7), 16–30.
Siering, M., & Muntermann, J. (2013). What drives the helpfulness of online product reviews? From stars to facts and emotions. In 11th international conference on Wirtschaft sinformatik.
Wang, C. C., Li, M. Z., & Yang, Y. H. (2015). Perceived usefulness of word-of-mouth: An analysis of sentimentality in product reviews (pp. 448–459). Berlin: Springer.
Wu, P. F. (2013). In search of negativity bias: An empirical study of perceived helpfulness of online reviews. Psychology & Marketing, 30(11), 971–984.
Chen, X., Sheng, J., Wang, X., & Deng, J. S. (2016). Exploring determinants of attraction and helpfulness of online product review: A consumer behavior perspective. Discrete Dynamics in Nature and Society, 1, 1–19.
Lee, S., & Choeh, J. Y. (2017). Exploring the determinants of and predicting the helpfulness of online user reviews. Management Decision, 40(3), 316–332.
Karimi, S., & Wang, F. (2017). Online review helpfulness: Impact of reviewer profile image. Decision Support Systems, 96, 39–48.
Yu, X., Liu, Y., Huang, X., & An, A. (2010). Mining online reviews for predicting sales performance: A case study in the movie domain. IEEE Transactions on Knowledge and Data Engineering, 24(4), 720–734.
Hong, H., & Xu, D. (2015). Research of online review helpfulness based on negative binary regress model the mediator role of reader participation. In International conference on service systems and service management (pp. 1–5). IEEE.
Filieri, R. (2015). What makes online reviews helpful? A diagnosticity-adoption framework to explain informational and normative influences in e-WOM. Journal of Business Research, 68(6), 1261–1270.
Ullah, R., Zeb, A., & Kim, W. (2015). The impact of emotions on the helpfulness of movie reviews. Journal of Applied Research & Technology, 13(3), 359–363.
Cheng, Y. H., & Ho, H. Y. (2015). Social influence’s impact on reader perceptions of online reviews. Journal of Business Research, 68(4), 883–887.
Hu, Y. H., & Chen, K. (2016). Predicting hotel review helpfulness: The impact of review visibility, and interaction between hotel stars and review ratings. International Journal of Information Management, 36(6), 929–944.
Fang, B., Ye, Q., Kucukusta, D., & Law, R. (2016). Analysis of the perceived value of online tourism reviews: Influence of readability and reviewer characteristics. Tourism Management, 52, 498–506.
Kwok, L., & Xie, K. L. (2016). Factors contributing to the helpfulness of online hotel reviews: does manager response play a role? International Journal of Contemporary Hospitality Management, 28(10), 2156–2177.
Yin, G., Zhang, Q., & Li, Y. (2014). Effects of emotional valence and arousal on consumer perceptions of online review helpfulness. In Twentieth Americas conference on information systems, Savannah.
Li, H., Zhang, Z., Janakiraman, R. & Fang, M. (2016) How review sentiment and readability affect online peer evaluation votes?—An examination combining reviewer’s social identity and social network. In Tourism travel and research association: Advancing tourism research globally (p. 29).
Ghose, A., & Ipeirotis, P. G. (2007). Designing novel review ranking systems: Predicting the usefulness and impact of reviews. In International conference on electronic commerce (pp. 303–310).
Zhang, Y. (2014). Automatically predicting the helpfulness of online reviews. In The fifteenth IEEE international conference on information reuse and integration (pp. 662–668). IEEE.
Acknowledgements
The work described in this paper is supported by National Natural Science Foundation of China (Grant Nos. 71531001 and 71572006).
Author information
Authors and Affiliations
Corresponding author
Appendix A
Appendix A
See Table 6.
Rights and permissions
About this article
Cite this article
Wang, Y., Wang, J. & Yao, T. What makes a helpful online review? A meta-analysis of review characteristics. Electron Commer Res 19, 257–284 (2019). https://doi.org/10.1007/s10660-018-9310-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10660-018-9310-2