Opinion Analysis Across Languages: An Overview of and Observations from the NTCIR6 Opinion Analysis Pilot Task

  • David Kirk Evans
  • Lun-Wei Ku
  • Yohei Seki
  • Hsin-Hsi Chen
  • Noriko Kando
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4578)


In this paper we introduce the NTCIR6 Opinion Analysis Pilot Task, information about the Chinese, Japanese, and English data, plans for future opinion analysis tasks at NTCIR, and a brief overview of the evaluation results. This pilot task is a sentence-level opinion identification and polarity detection task run over data from a comparable corpus in three languages: Chinese, English, and Japanese. We have manually annotated documents for this task in each language, producing what we believe to be the first multilingual opinion analysis data set over comparable data. Six participants submitted Chinese system results, three Japanese, and six English for this pilot task. We plan to release the data to the research community, and hope to spur further research into cross-lingual opinion analysis and its use in other NLP tasks. In particular, we look forward to researchers using this data to investigate cross-cultural perspective differences based on automatic sentiment analysis.


Natural Language Processing Machine Translation Computational Linguistics Daily News Relevance Judgment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hatzivassiloglou, V., McKeown, K.R.: Predicting the semantic orientation of adjectives. In: Proceedings of the eighth conference on European chapter of the Association for Computational Linguistics, Morristown, NJ, USA, pp. 174–181 (1997)Google Scholar
  2. 2.
    Hatzivassiloglou, V., Wiebe, J.M.: Effects of adjective orientation and gradability on sentence subjectivity. In: Proceedings of the 18th International Conference on Computational Linguistics (2000)Google Scholar
  3. 3.
    Kanayama, H., Nasukawa, T.: Deeper sentiment analysis using machine translation technology. In: Proceedings of the 20th International Conference on Computational Linguistics (COLING), pp. 494–500 (2004)Google Scholar
  4. 4.
    Kanayama, H., Nasukawa, T.: Fully automatic lexicon expansion for domain-oriented sentiment analysis. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, pp. 355–363 (July 2006)Google Scholar
  5. 5.
    Kando, N., Kuriyama, K., Nozue, T., Eguchi, K., Karo, H., Hidaka, S., Adachi, J.: The ntcir workshop: the first evaluation workshop on japanese text retrieval and cross-lingual information retrieval. In: Proceedings of the 4th International Workshop on Information Retrieval with Asian Languages (1RAL 1999) (1999)Google Scholar
  6. 6.
    Kishida, K., Hua Chen, K., Lee, S., Kuriyama, K., Kando, N., Chen, H.-H., Myaeng, S.H.: Overview of clir task at the fifth ntcir workshop. In: Proceedings of the Fifth NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, Question Answering and Cross-Lingual Information Access, Tokyo, Japan (December 2005) (National Institute of Informatics) (2005)Google Scholar
  7. 7.
    Ku, L.-W., Liang, Y.-T., Chen, H.-H.: Opinion extraction, summarization and tracking in news and blog corpora. In: Proceedings of AAAI-2006 Spring Symposium on Computational Approaches to Analyzing Weblogs, AAAI Technical Report (2006)Google Scholar
  8. 8.
    Pang, B., Lee, L.: Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In: Proceedings of the ACL, pp. 115–124 (2005)Google Scholar
  9. 9.
    Riloff, E., Wiebe, J.: Learning extraction patterns for subjective expressions. In: Proceedings of the 2003 conference on Empirical methods in natural language processing (EMNLP 2003), Sapporo, Japan, pp. 105–112 (July 2003)Google Scholar
  10. 10.
    Seki, Y., Eguchi, K., Kando, N.: Multi-document viewpoint summarization focused on facts, opinion and knowledge (chapter 24). In: Shanahan, J.G., Qu, Y., Wiebe, J. (eds.) Computing Attitude and Affect in Text: Theories and Applications, pp. 317–336. Springer, Dordrecht, The Netherlands (2005)Google Scholar
  11. 11.
    Seki, Y., Eguchi, K., Kando, N., Aono, M.: Opinion-focused Summarization and its Analysis at DUC 2006. In: Proc. of the Document Understanding Conf. Wksp. 2005 (DUC 2006) at the Human Language Technology Conf. - North American chapter of the Association for Computational Linguistics (HLT-NAACL 2006),New York Marriott pp. 122–130 (June 2006)Google Scholar
  12. 12.
    Seki, Y., Evans, D.K., Ku, L.-W., Chen, H.-H., Kando, N., Lin, C.-Y.: Overview of opinion analysis pilot task at ntcir-6. In: Proceedings of the Sixth NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, Question Answering and Cross-Lingual Information Access National Institute of Informatics (May 2007)Google Scholar
  13. 13.
    Wiebe, J., Bruce, R., O’Hara, T.: Development and use of a gold-standard data set for subjectivity classifications. In: Proceedings of the 37th Association of Computational Linguistics, pp. 246–253 (1999)Google Scholar
  14. 14.
    Yi, J., Nasukawa, T., Bunescu, R., Niblack, W.: Sentiment analyzer: Extracting sentiments about a given topic using natural language processing techniques. In: The Third IEEE International Conference on Data Mining, November 2003, pp. 427–343. IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  15. 15.
    Yu, H., Hatzivassiloglou, V.: Towards answering opinion questions: separating facts from opinions and identifying the polarity of opinion sentences. In: Proceedings of the conference on Empirical methods in natural language processing, (Association for Computational Linguistics), Morristown, NJ, USA (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • David Kirk Evans
    • 1
  • Lun-Wei Ku
    • 3
  • Yohei Seki
    • 2
  • Hsin-Hsi Chen
    • 3
  • Noriko Kando
    • 1
  1. 1.National Institute of Informatics, TokyoJapan
  2. 2.Dept. of Information and Computer Sciences, Toyohashi University of TechnologyJapan
  3. 3.Department of Computer Science and Information Engineering, National Taiwan University, TaipeiTaiwan

Personalised recommendations