Skip to main content
Log in

StarTrack: The Next Generation (of Product Review Management Tools)

New Generation Computing Aims and scope Submit manuscript

Abstract

Online product reviews are increasingly being recognized as a gold mine of information for determining product and brand positioning, and more and more companies look for ways of digging this gold mine for nuggets of knowledge that they can then bring to bear in decision making. We present a software system, called StarTrack, that automatically rates a product review according to a number of “stars,” i.e., according to how positive it is. In other words, given a text-only review (i.e., one with no explicit star-rating attached), StarTrack attempts to guess the star-rating that the reviewer would have attached to the review. StarTrack is thus useful for analysing unstructured word-of-mouth on products, such as the comments and reviews about products that are to be found in spontaneous discussion forums, such as newsgroups, blogs, and the like. StarTrack is based on machine learning technology, and as such does not require any re-programming for porting it from one product domain to another. Based on the star-ratings it attributes to reviews, StarTrack can subsequently rank the products in a given set according to how favourably they have been reviewed by consumers. We present controlled experiments in which we evaluate, on two large sets of product reviews crawled from the Web, the accuracy of StarTrack at (i) star-rating reviews, and (ii) ranking the reviewed products based on the automatically attributed star-ratings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aiolli F., Cardin R., Sebastiani F., Sperduti A.: “Preferential text classification: Learning algorithms and evaluation measures,”. Information Retrieval 12(5), 559–580 (2009)

    Article  Google Scholar 

  2. Argamon, S., Bloom, K., Esuli, A. and Sebastiani, F., “Automatically determining attitude type and force for sentiment analysis,” in Proc. of the 3rd Language Technology Conference (LTC 2007), pp. 369–373, Poznań, PL, 2007.

  3. Baccianella, S., Esuli, A. and Sebastiani, F., “Evaluation measures for ordinal text classification,” in Proc. of the 9th IEEE International Conference on Intelligent Systems Design and Applications (ISDA 2009), pp. 283–287, Pisa, IT, 2009.

  4. Baccianella, S., Esuli, A. and Sebastiani, F., “Multi-facet rating of product reviews,” in Proc. of the 31st European Conference on Information Retrieval (ECIR 2009), pp. 461–472, Toulouse, FR, 2009.

  5. Baccianella, S., Esuli, A. and Sebastiani, F., “Feature selection for ordinal regression,” in Proc. of the 25th ACM Symposium on Applied Computing (SAC 2010), pp. 1748–1754, Sierre, CH, 2010.

  6. Baccianella, S., Esuli, A. and Sebastiani, F., “SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining,” in Proc. of the 7th Conference on Language Resources and Evaluation (LREC 2010), Valletta, MT, 2010.

  7. Blitzer, J., Dredze, M. and Pereira, F., “Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification,” in Proc. of the 45th Annual Meeting of the Association for Computational Linguistics (ACL 2007), pp. 440–447, Prague, CZ, 2007.

  8. Chang, C.-C. and Lin, C.-J., LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~libsvm.

  9. Chen C.C., Tseng Y.-D.: “Quality evaluation of product reviews using an information quality framework,”. Decision Support Systems 50(4), 755–768 (2011)

    Article  Google Scholar 

  10. Chen Y., Xie J.: “Third-party product review and firm marketing strategy,”. Marketing Science 23(2), 218–240 (2005)

    Article  Google Scholar 

  11. Chen Y., Xie J.: “Online consumer review: Word-of-mouth as a new element of marketing communication mix,”. Management Science 54(3), 477–491 (2008)

    Article  Google Scholar 

  12. Chevalier J.A., Mayzlin D.: “The effect of word of mouth on sales: Online book reviews,”. Journal of Marketing Research 43(3), 345–354 (2006)

    Article  Google Scholar 

  13. Chu W., Keerthi S.S.: “Support vector ordinal regression,”. Neural Computation 19(3), 145–152 (2007)

    Article  MathSciNet  Google Scholar 

  14. Dang Y., Zhang Y., Chen H.: “A lexicon-enhanced method for sentiment classification: An experiment on online product reviews,”. IEEE Intelligent Systems 25(4), 46–53 (2010)

    Article  Google Scholar 

  15. Dave, K., Lawrence, S. and Pennock, D. M., “Mining the peanut gallery: Opinion extraction and semantic classification of product reviews,” in Proc. of the 12th International Conference on the World Wide Web (WWW 2003), pp. 519– 528, Budapest, HU, 2003.

  16. Dellarocas C., Zhang X., Awad N.F.: “Exploring the value of online product reviews in forecasting sales: The case of motion pictures,”. Journal of Interactive Marketing 21(4), 23–45 (2007)

    Article  Google Scholar 

  17. Drucker, H., Burges, C. J., Kaufman, L., Smola, A. and Vapnik, V., “Support vector regression machines,” in Proc. of the 9th Conference on Neural Information Processing Systems (NIPS 1996), pp. 155–161, Denver, US, 1997.

  18. Fagin, R., Kumar, R., Mahdiany, M., Sivakumar, D. and Veez, E., “Comparing and aggregating rankings with ties,” in Proc. of ACM International Conference on Principles of Database Systems (PODS 2004), pp. 47–58, Paris, FR, 2004.

  19. Fellbaum, C. ed., WordNet: An Electronic Lexical Database, MIT Press, Cambridge, US, 1998.

  20. Gao, G., Gu, B. and Lin, M., “The dynamics of online consumer reviews,” in Proc. of the Workshop on Information Systems and Economics (WISE 2006), Evanston, US, 2006.

  21. Goldberg, A. B. and Zhu, X., “Seeing stars when there aren’t many stars: Graphbased semi-supervised learning for sentiment categorization,” in Proc. of the HLT/NAACL Workshop on Graph-based Algorithms for Natural Language Processing, pp. 45–52, New York, US, 2006.

  22. Gretzel, U. and Yoo, K. Y., “Use and impact of online travel review,” in Proc. of the 2008 International Conference on Information and Communication Technologies in Tourism, pp. 35–46, Innsbruck, AT, 2008.

  23. Hu, N., Zhang, J. and Pavlou, P. A., “Overcoming the J-shaped distribution of product reviews,” Communications of the ACM, 52, 10, pp. 144–147, 2009.

  24. Jindal, N. and Liu, B., “Review spam detection,” in Proc. of the 16th International Conference on the World Wide Web (WWW 2007), pp. 1189–1190, Banff, CA, 2007.

  25. Kendall M.: “A new measure of rank correlation,”. Biometrika 30, 81–89 (1938)

    MathSciNet  MATH  Google Scholar 

  26. Kennedy A., Inkpen D.: “Sentiment classification of movie reviews using contextual valence shifters,”. Computational Intelligence 22(2), 110–125 (2006)

    Article  MathSciNet  Google Scholar 

  27. Krippendorff, K., Content analysis: An introduction to its methodology, Sage, Thousand Oaks, US, 2004.

  28. Lim, E.-P., Nguyen, V.-A., Jindal, N., Liu, B. and Lauw, H. W., “Detecting product review spammers using rating behaviors,” in Proc. of the 19th ACM International Conference on Information and Knowledge Management (CIKM 2010), pp. 939–948, Toronto, CA, 2010.

  29. Martin, J. R. and White, P. R., The Language of Evaluation: Appraisal in English, Palgrave, London, UK, 2005.

  30. Min, H.-J. and Park, J. C., “Toward finer-grained sentiment identification in product reviews through linguistic and ontological analyses,” in Proc. of the 47th Annual Meeting of the Association for Computational Linguistics (ACL 2009), pp. 169–172, Singapore, SN, 2009.

  31. O’Mahony, M. P. and Smyth, B., “Using readability tests to predict helpful product reviews,” in Proc. of the 9th International Conference on “Recherche d’ Information Assistée par Ordinateur” (RIAO 2010), pp. 164–167, Paris, FR, 2010.

  32. Ott, M., Choi, Y., Cardie, C. and Hancock, J. T., “Finding deceptive opinion spam by any stretch of the imagination,” in Proc. of the 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011), pp. 309–319, Portland, US, 2011.

  33. Pang, B. and Lee, L., “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,” in Proc. of the 43rd Meeting of the Association for Computational Linguistics (ACL 2005), pp. 115–124, Ann Arbor, US, 2005.

  34. Pang, B. and Lee, L., “Opinion mining and sentiment analysis,” Foundations and Trends in Information Retrieval, 2, 1/2, pp. 1–135, 2008.

  35. Pekar, V. and Ou, S., “Discovery of subjective evaluations of product features in hotel reviews,” Journal of Vacation Marketing, 14, 2, pp. 145–156, 2008.

  36. Popescu, A.-M. and Etzioni, O., “Extracting product features and opinions from reviews,” in Proc. of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP 2005), pp. 339–346, Vancouver, CA, 2005.

  37. Schölkopf B., Smola A.J., Williamson R.C., Bartlett P.L.: “New support vector algorithms,”. Neural Computation 12(5), 1207–1245 (2000)

    Article  Google Scholar 

  38. Sénécal S., Nantel J.: “The influence of online product recommendations on consumers’ online choices,”. Journal of Retailing 80, 159–169 (2004)

    Article  Google Scholar 

  39. Shimada, K. and Endo, T., “Seeing several stars: A rating inference task for a document containing several evaluation criteria,” in Proc. of the 12th Pacific- Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2008), pp. 1006–1014, Osaka, JP, 2008.

  40. Stone, P. J., Dunphy, D. C., Smith, M. S. and Ogilvie, D. M., The General Inquirer: A Computer Approach to Content Analysis, The MIT Press, Cambridge, US, 1966.

  41. Wei, W., Gulla, J. A. and Fu, Z., “Enhancing negation-aware sentiment classification on product reviews via multi-unigram feature generation,” in Proc. of the 6th International Conference on Intelligent Computing (ICIC 2010), pp. 380–391, Changsha, CN, 2010.

  42. Yang J.-Y., Kim H.-j., Lee S.-g.: “Feature-based product review summarization utilizing user score,”. Journal of Information Science and Engineering 26(6), 1973–1990 (2010)

    MathSciNet  Google Scholar 

  43. Zhan J., Loh H.T., Liu Y.: “Gather customer concerns from online product reviews - a text summarization approach,”. Expert Systems with Applications 36(2), 2107–2115 (2009)

    Article  Google Scholar 

  44. Zhang R., Tran T.T.: “An information gain-based approach for recommending useful product reviews,”. Knowledge and Information Systems 26(3), 419–434 (2011)

    Article  Google Scholar 

  45. Zhang, Z. and Varadarajan, B., “Utility scoring of product reviews,” in Proc. of the 15th ACM International Conference on Information and Knowledge Management (CIKM 2006), pp. 51–57, Arlington, US, 2006.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Stefano Baccianella.

About this article

Cite this article

Baccianella, S., Esuli, A. & Sebastiani, F. StarTrack: The Next Generation (of Product Review Management Tools). New Gener. Comput. 31, 47–70 (2013). https://doi.org/10.1007/s00354-012-0122-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-012-0122-y

Keywords

Navigation