BERT Feature Based Model for Predicting the Helpfulness Scores of Online Customers Reviews

  • Shuzhe XuEmail author
  • Salvador E. Barbosa
  • Don Hong
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1130)


Online product reviews help consumers make purchase decisions when shopping online. As such, many computational models have been constructed to automatically evaluate the helpfulness of customer product reviews. However, many existing models are based on simple explanatory variables, including those extracted from low quality reviews that can be misleading and lead to confusion. Quality feature selection is essential for predicting the helpfulness of online customer reviews. The Bidirectional Encoder Representations from Transformers (BERT) is a very recently developed language representation model which can attain state-of-the-art results on many natural language processing tasks. In this study, a predictive model for determining helpfulness scores of customer reviews based on incorporation of BERT features with deep learning techniques is proposed. The application analyzes the Amazon product reviews dataset, and uses a BERT features based algorithm expected to be useful in help consumers to make a better purchase decisions.


BERT Online review Review helpfulness Neural network Data mining Prediction model 



The authors are indebted to anonymous reviewers for providing constructive comments and suggestions which has resulted in improvement both the readability and quality of the paper.


  1. 1.
    Liu, Y., Huang, X., An, A., Yu, X.: Modeling and predicting the helpfulness of online reviews. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 443–452. IEEE (2008)Google Scholar
  2. 2.
    Zhang, Z., Varadarajan, B.: Utility scoring of product reviews. In: Proceedings of the 15th ACM International Conference on Information and Knowledge Management, pp. 51–57. ACM (2006)Google Scholar
  3. 3.
    Mudambi, S.M., Schuff, D.: What makes a helpful review? A study of customer reviews on MIS Q. 34(1), 185–200 (2010)CrossRefGoogle Scholar
  4. 4.
    Park, Y.J.: Predicting the helpfulness of online customer reviews across different product types. Sustainability 10(6), 1735 (2018)CrossRefGoogle Scholar
  5. 5.
    Dave, K., Lawrence, S., Pennock, D.M.: Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In: Proceedings of the 12th International Conference on World Wide Web, pp. 519–528. ACM (2003)Google Scholar
  6. 6.
    Read, J.: Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In: Proceedings of the ACL Student Research Workshop, pp. 43–48. Association for Computational Linguistics (2005)Google Scholar
  7. 7.
    Ye, Q., Zhang, Z., Law, R.: Sentiment classification of online reviews to travel destinations by supervised machine learning approaches. Expert Syst. Appl. 36(3), 6527–6535 (2009)CrossRefGoogle Scholar
  8. 8.
    Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805 (2018)
  9. 9.
    Xu, H., Liu, B., Shu, L., Yu, P.S.: BERT post-training for review reading comprehension and aspect-based sentiment analysis. arXiv preprint arXiv:190402232 (2019)
  10. 10.
    Eisend, M.: Two-sided advertising: a meta-analysis. Int. J. Res. Mark. 23(2), 187–198 (2006)CrossRefGoogle Scholar
  11. 11.
    Pavlou, P.A., Dimoka, A.: The nature and role of feedback text comments in online marketplaces: implications for trust building, price premiums, and seller differentiation. Inf. Syst. Res. 17(4), 392–414 (2006)CrossRefGoogle Scholar
  12. 12.
    Forman, C., Ghose, A., Wiesenfeld, B.: Examining the relationship between reviews and sales: the role of reviewer identity disclosure in electronic markets. Inf. Syst. Res. 19(3), 291–313 (2008)CrossRefGoogle Scholar
  13. 13.
    Nelson, P.: Information and consumer behavior. J. Polit. Econ. 78(2), 311–329 (1970)CrossRefGoogle Scholar
  14. 14.
    Ford, G.T., Smith, D.B., Swasy, J.L.: Consumer skepticism of advertising claims: testing hypotheses from economics of information. J. Consum. Res. 16(4), 433–441 (1990)CrossRefGoogle Scholar
  15. 15.
    Pan, Y., Zhang, J.Q.: Born unequal: a study of the helpfulness of user-generated product reviews. J. Retail. 87(4), 598–612 (2011)CrossRefGoogle Scholar
  16. 16.
    Ghose, A., Ipeirotis, P.G.: Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Trans. Knowl. Data Eng. 23(10), 1498–1512 (2010)CrossRefGoogle Scholar
  17. 17.
    Korfiatis, N., García-Bariocanal, E., Sánchez-Alonso, S.: Evaluating content quality and helpfulness of online product reviews: the interplay of review helpfulness vs. review content. Electron. Commer. Res. Appl. 11(3), 205–217 (2012)CrossRefGoogle Scholar
  18. 18.
    Cao, Q., Duan, W., Gan, Q.: Exploring determinants of voting for the helpfulness of online user reviews: a text mining approach. Decis. Support Syst. 50(2), 511–521 (2011)CrossRefGoogle Scholar
  19. 19.
    Kuan, K.K., Hui, K.L., Prasarnphanich, P., Lai, H.Y.: What makes a review voted? An empirical investigation of review voting in online review systems. J. Assoc. Inf. Syst. 16(1), 48 (2015)Google Scholar
  20. 20.
    Yin, D., Bond, S., Zhang, H.: Anxious or angry? Effects of discrete emotions on the perceived helpfulness of online reviews. MIS Q. 38(2), 539–560 (2014)CrossRefGoogle Scholar
  21. 21.
    Ahmad, S.N., Laroche, M.: How do expressed emotions affect the helpfulness of a product review? Evidence from reviews using latent semantic analysis. Int. J. Electron. Commer. 20(1), 76–111 (2015)CrossRefGoogle Scholar
  22. 22.
    Ma, Y., Xiang, Z., Du, Q., Fan, W.: Effects of user-provided photos on hotel review helpfulness: an analytical approach with deep leaning. Int. J. Hosp. Manag. 71, 120–131 (2018)CrossRefGoogle Scholar
  23. 23.
    Chen, C., Yang, Y., Zhou, J., Li, X., Bao, F.S.: Cross-domain review helpfulness prediction based on convolutional neural networks with auxiliary domain discriminators. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Short Papers), vol. 2, pp. 602–607 (2018)Google Scholar
  24. 24.
    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)Google Scholar
  25. 25.
    Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., Zettlemoyer, L.: Deep contextualized word representations. arXiv preprint arXiv:180205365 (2018)
  26. 26.
    Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. arXiv preprint arXiv:180106146 (2018)
  27. 27.
    Bragge, J., Storgårds, J.: Utilizing text-mining tools to enrich traditional literature reviews. case: digital games. In: Proceedings of the 30th Information Systems Research Seminar in Scandinavia IRIS, pp. 1–24 (2007)Google Scholar
  28. 28.
    Weathers, D., Sharma, S., Wood, S.L.: Effects of online communication practices on consumer perceptions of performance uncertainty for search and experience goods. J. Retail. 83(4), 393–401 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Computational Science ProgramMiddle Tennessee State UniversityMurfreesboroUSA

Personalised recommendations