New Generation Computing

, Volume 31, Issue 1, pp 47–70 | Cite as

StarTrack: The Next Generation (of Product Review Management Tools)

  • Stefano Baccianella
  • Andrea Esuli
  • Fabrizio Sebastiani


Online product reviews are increasingly being recognized as a gold mine of information for determining product and brand positioning, and more and more companies look for ways of digging this gold mine for nuggets of knowledge that they can then bring to bear in decision making. We present a software system, called StarTrack, that automatically rates a product review according to a number of “stars,” i.e., according to how positive it is. In other words, given a text-only review (i.e., one with no explicit star-rating attached), StarTrack attempts to guess the star-rating that the reviewer would have attached to the review. StarTrack is thus useful for analysing unstructured word-of-mouth on products, such as the comments and reviews about products that are to be found in spontaneous discussion forums, such as newsgroups, blogs, and the like. StarTrack is based on machine learning technology, and as such does not require any re-programming for porting it from one product domain to another. Based on the star-ratings it attributes to reviews, StarTrack can subsequently rank the products in a given set according to how favourably they have been reviewed by consumers. We present controlled experiments in which we evaluate, on two large sets of product reviews crawled from the Web, the accuracy of StarTrack at (i) star-rating reviews, and (ii) ranking the reviewed products based on the automatically attributed star-ratings.


Product reviews Sentiment analysis Sentiment lexicons Ordinal regression Text classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aiolli F., Cardin R., Sebastiani F., Sperduti A.: “Preferential text classification: Learning algorithms and evaluation measures,”. Information Retrieval 12(5), 559–580 (2009)CrossRefGoogle Scholar
  2. 2.
    Argamon, S., Bloom, K., Esuli, A. and Sebastiani, F., “Automatically determining attitude type and force for sentiment analysis,” in Proc. of the 3rd Language Technology Conference (LTC 2007), pp. 369–373, Poznań, PL, 2007.Google Scholar
  3. 3.
    Baccianella, S., Esuli, A. and Sebastiani, F., “Evaluation measures for ordinal text classification,” in Proc. of the 9th IEEE International Conference on Intelligent Systems Design and Applications (ISDA 2009), pp. 283–287, Pisa, IT, 2009.Google Scholar
  4. 4.
    Baccianella, S., Esuli, A. and Sebastiani, F., “Multi-facet rating of product reviews,” in Proc. of the 31st European Conference on Information Retrieval (ECIR 2009), pp. 461–472, Toulouse, FR, 2009.Google Scholar
  5. 5.
    Baccianella, S., Esuli, A. and Sebastiani, F., “Feature selection for ordinal regression,” in Proc. of the 25th ACM Symposium on Applied Computing (SAC 2010), pp. 1748–1754, Sierre, CH, 2010.Google Scholar
  6. 6.
    Baccianella, S., Esuli, A. and Sebastiani, F., “SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining,” in Proc. of the 7th Conference on Language Resources and Evaluation (LREC 2010), Valletta, MT, 2010.Google Scholar
  7. 7.
    Blitzer, J., Dredze, M. and Pereira, F., “Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification,” in Proc. of the 45th Annual Meeting of the Association for Computational Linguistics (ACL 2007), pp. 440–447, Prague, CZ, 2007.Google Scholar
  8. 8.
    Chang, C.-C. and Lin, C.-J., LIBSVM: a library for support vector machines, 2001. Software available at
  9. 9.
    Chen C.C., Tseng Y.-D.: “Quality evaluation of product reviews using an information quality framework,”. Decision Support Systems 50(4), 755–768 (2011)CrossRefGoogle Scholar
  10. 10.
    Chen Y., Xie J.: “Third-party product review and firm marketing strategy,”. Marketing Science 23(2), 218–240 (2005)CrossRefGoogle Scholar
  11. 11.
    Chen Y., Xie J.: “Online consumer review: Word-of-mouth as a new element of marketing communication mix,”. Management Science 54(3), 477–491 (2008)CrossRefGoogle Scholar
  12. 12.
    Chevalier J.A., Mayzlin D.: “The effect of word of mouth on sales: Online book reviews,”. Journal of Marketing Research 43(3), 345–354 (2006)CrossRefGoogle Scholar
  13. 13.
    Chu W., Keerthi S.S.: “Support vector ordinal regression,”. Neural Computation 19(3), 145–152 (2007)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Dang Y., Zhang Y., Chen H.: “A lexicon-enhanced method for sentiment classification: An experiment on online product reviews,”. IEEE Intelligent Systems 25(4), 46–53 (2010)CrossRefGoogle Scholar
  15. 15.
    Dave, K., Lawrence, S. and Pennock, D. M., “Mining the peanut gallery: Opinion extraction and semantic classification of product reviews,” in Proc. of the 12th International Conference on the World Wide Web (WWW 2003), pp. 519– 528, Budapest, HU, 2003.Google Scholar
  16. 16.
    Dellarocas C., Zhang X., Awad N.F.: “Exploring the value of online product reviews in forecasting sales: The case of motion pictures,”. Journal of Interactive Marketing 21(4), 23–45 (2007)CrossRefGoogle Scholar
  17. 17.
    Drucker, H., Burges, C. J., Kaufman, L., Smola, A. and Vapnik, V., “Support vector regression machines,” in Proc. of the 9th Conference on Neural Information Processing Systems (NIPS 1996), pp. 155–161, Denver, US, 1997.Google Scholar
  18. 18.
    Fagin, R., Kumar, R., Mahdiany, M., Sivakumar, D. and Veez, E., “Comparing and aggregating rankings with ties,” in Proc. of ACM International Conference on Principles of Database Systems (PODS 2004), pp. 47–58, Paris, FR, 2004.Google Scholar
  19. 19.
    Fellbaum, C. ed., WordNet: An Electronic Lexical Database, MIT Press, Cambridge, US, 1998.Google Scholar
  20. 20.
    Gao, G., Gu, B. and Lin, M., “The dynamics of online consumer reviews,” in Proc. of the Workshop on Information Systems and Economics (WISE 2006), Evanston, US, 2006.Google Scholar
  21. 21.
    Goldberg, A. B. and Zhu, X., “Seeing stars when there aren’t many stars: Graphbased semi-supervised learning for sentiment categorization,” in Proc. of the HLT/NAACL Workshop on Graph-based Algorithms for Natural Language Processing, pp. 45–52, New York, US, 2006.Google Scholar
  22. 22.
    Gretzel, U. and Yoo, K. Y., “Use and impact of online travel review,” in Proc. of the 2008 International Conference on Information and Communication Technologies in Tourism, pp. 35–46, Innsbruck, AT, 2008.Google Scholar
  23. 23.
    Hu, N., Zhang, J. and Pavlou, P. A., “Overcoming the J-shaped distribution of product reviews,” Communications of the ACM, 52, 10, pp. 144–147, 2009.Google Scholar
  24. 24.
    Jindal, N. and Liu, B., “Review spam detection,” in Proc. of the 16th International Conference on the World Wide Web (WWW 2007), pp. 1189–1190, Banff, CA, 2007.Google Scholar
  25. 25.
    Kendall M.: “A new measure of rank correlation,”. Biometrika 30, 81–89 (1938)MathSciNetzbMATHGoogle Scholar
  26. 26.
    Kennedy A., Inkpen D.: “Sentiment classification of movie reviews using contextual valence shifters,”. Computational Intelligence 22(2), 110–125 (2006)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Krippendorff, K., Content analysis: An introduction to its methodology, Sage, Thousand Oaks, US, 2004.Google Scholar
  28. 28.
    Lim, E.-P., Nguyen, V.-A., Jindal, N., Liu, B. and Lauw, H. W., “Detecting product review spammers using rating behaviors,” in Proc. of the 19th ACM International Conference on Information and Knowledge Management (CIKM 2010), pp. 939–948, Toronto, CA, 2010.Google Scholar
  29. 29.
    Martin, J. R. and White, P. R., The Language of Evaluation: Appraisal in English, Palgrave, London, UK, 2005.Google Scholar
  30. 30.
    Min, H.-J. and Park, J. C., “Toward finer-grained sentiment identification in product reviews through linguistic and ontological analyses,” in Proc. of the 47th Annual Meeting of the Association for Computational Linguistics (ACL 2009), pp. 169–172, Singapore, SN, 2009.Google Scholar
  31. 31.
    O’Mahony, M. P. and Smyth, B., “Using readability tests to predict helpful product reviews,” in Proc. of the 9th International Conference on “Recherche d’ Information Assistée par Ordinateur” (RIAO 2010), pp. 164–167, Paris, FR, 2010.Google Scholar
  32. 32.
    Ott, M., Choi, Y., Cardie, C. and Hancock, J. T., “Finding deceptive opinion spam by any stretch of the imagination,” in Proc. of the 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011), pp. 309–319, Portland, US, 2011.Google Scholar
  33. 33.
    Pang, B. and Lee, L., “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,” in Proc. of the 43rd Meeting of the Association for Computational Linguistics (ACL 2005), pp. 115–124, Ann Arbor, US, 2005.Google Scholar
  34. 34.
    Pang, B. and Lee, L., “Opinion mining and sentiment analysis,” Foundations and Trends in Information Retrieval, 2, 1/2, pp. 1–135, 2008.Google Scholar
  35. 35.
    Pekar, V. and Ou, S., “Discovery of subjective evaluations of product features in hotel reviews,” Journal of Vacation Marketing, 14, 2, pp. 145–156, 2008.Google Scholar
  36. 36.
    Popescu, A.-M. and Etzioni, O., “Extracting product features and opinions from reviews,” in Proc. of the Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP 2005), pp. 339–346, Vancouver, CA, 2005.Google Scholar
  37. 37.
    Schölkopf B., Smola A.J., Williamson R.C., Bartlett P.L.: “New support vector algorithms,”. Neural Computation 12(5), 1207–1245 (2000)CrossRefGoogle Scholar
  38. 38.
    Sénécal S., Nantel J.: “The influence of online product recommendations on consumers’ online choices,”. Journal of Retailing 80, 159–169 (2004)CrossRefGoogle Scholar
  39. 39.
    Shimada, K. and Endo, T., “Seeing several stars: A rating inference task for a document containing several evaluation criteria,” in Proc. of the 12th Pacific- Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2008), pp. 1006–1014, Osaka, JP, 2008.Google Scholar
  40. 40.
    Stone, P. J., Dunphy, D. C., Smith, M. S. and Ogilvie, D. M., The General Inquirer: A Computer Approach to Content Analysis, The MIT Press, Cambridge, US, 1966.Google Scholar
  41. 41.
    Wei, W., Gulla, J. A. and Fu, Z., “Enhancing negation-aware sentiment classification on product reviews via multi-unigram feature generation,” in Proc. of the 6th International Conference on Intelligent Computing (ICIC 2010), pp. 380–391, Changsha, CN, 2010.Google Scholar
  42. 42.
    Yang J.-Y., Kim H.-j., Lee S.-g.: “Feature-based product review summarization utilizing user score,”. Journal of Information Science and Engineering 26(6), 1973–1990 (2010)MathSciNetGoogle Scholar
  43. 43.
    Zhan J., Loh H.T., Liu Y.: “Gather customer concerns from online product reviews - a text summarization approach,”. Expert Systems with Applications 36(2), 2107–2115 (2009)CrossRefGoogle Scholar
  44. 44.
    Zhang R., Tran T.T.: “An information gain-based approach for recommending useful product reviews,”. Knowledge and Information Systems 26(3), 419–434 (2011)CrossRefGoogle Scholar
  45. 45.
    Zhang, Z. and Varadarajan, B., “Utility scoring of product reviews,” in Proc. of the 15th ACM International Conference on Information and Knowledge Management (CIKM 2006), pp. 51–57, Arlington, US, 2006.Google Scholar

Copyright information

© Ohmsha and Springer Japan 2013

Authors and Affiliations

  • Stefano Baccianella
    • 1
  • Andrea Esuli
    • 1
  • Fabrizio Sebastiani
    • 1
  1. 1.Istituto di Scienza e Tecnologia dell’InformazioneConsiglio Nazionale delle RicerchePisaItaly

Personalised recommendations