Seeing Several Stars: A Rating Inference Task for a Document Containing Several Evaluation Criteria

  • Kazutaka Shimada
  • Tsutomu Endo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5012)


In this paper we address a novel sentiment analysis task of rating inference. Previous rating inference tasks, which are sometimes referred to as “seeing stars”, estimate only one rating in a document. However reviewers judge not only the overall polarity for a product but also details for it. A document in this new task contains several ratings for a product. Furthermore the range of the ratings is zero to six points (i.e., stars). In other words this task denotes “seeing several stars in a document”. If significant words or phrases for evaluation criteria and their strength as positive or negative opinions are extracted, a system with the knowledge can recommend products for users appropriately. For example, the system can output a detailed summary from review documents. In this paper we compare several methods to infer the ratings in a document and discuss a feature selection approach for the methods. The experimental results are useful for new researchers who try this new task.


Sentiment analysis Rating inference Review mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Berger, A.L., Della Pietra, S.A., Della Pietra, V.J.: A maximum entropy approach to natural language processing. Computational Linguistics 22(1), 39–71 (1996)Google Scholar
  2. 2.
    Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1), 37–46 (1960)CrossRefGoogle Scholar
  3. 3.
    Goldberg, A.B., Zhu, X.: Seeing stars when there aren’t many stars: Graph-based semi-supervised learning for sentiment categorization. In: HLT-NAACL 2006 Workshop on Textgraphs: Graph-based Algorithms for Natural Language Processing (2006)Google Scholar
  4. 4.
    Joachims, T.: Transductive inference for text classification using support vecor machines. In: Proceedings of the Sixteenth International Conference on Machine Learning, pp. 200–209 (1999)Google Scholar
  5. 5.
    Kawano, Y., Shimada, K., Endo, T.: Sentence polarity classification based on a scoring method (in Japanese). In: HINOKUNI Symposium 2007 CD-ROM A-3-4 (2007)Google Scholar
  6. 6.
    Kobayashi, N., Iida, R., Inui, K., Matsumoto, Y.: Opinion extraction using a learning-based anaphora resolution technique. In: Proceedings of the Second International Joint Conference on Natural Language Processing (IJCNLP-2005), pp. 175–180 (2005)Google Scholar
  7. 7.
    Koppel, M., Schler, J.: The importance of neutral examples in learning sentiment. Computational Intelligence 22(2), 100–109 (2006)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Okanohara, D., Tsujii, J.: Assigning polarity scores to reviews using machine learning techniques. In: Proceedings of the 2nd International Joint Conference on Natural Language Processing (IJCNLP), pp. 314–325 (2005)Google Scholar
  9. 9.
    Pang, B., Lee, L.: Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the 43th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 115–124 (2005)Google Scholar
  10. 10.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 79–86 (2002)Google Scholar
  11. 11.
    Tsutsumi, K., Shimada, K., Endo, T.: Movie review classification based on a multiple classifier. In: The 21th Pacific Asia Conference on Language, Information and Computation (PACLIC) (2007)Google Scholar
  12. 12.
    Turney, P.D.: Thumbs up? or thumbs down? semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 417–424 (2002)Google Scholar
  13. 13.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, Chichester (1999)Google Scholar
  14. 14.
    Wilson, T., Wiebe, J., Hwa, R.: Just how mad are you? finding strong and weak opinion clauses. In: AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications (2004)Google Scholar
  15. 15.
    Zhuang, L., Jing, F., Zhul, X.-Y.: Movie review mining and summarization. In: Proceedings of the ACM 15th Conference on Information and Knowledge Management (CIKM-2006), pp. 43–50 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Kazutaka Shimada
    • 1
  • Tsutomu Endo
    • 1
  1. 1.Department of Artificial IntelligenceKyushu Institute of TechnologyJapan

Personalised recommendations