Overview of NLPCC 2018 Shared Task 1: Emotion Detection in Code-Switching Text
This paper presents the overview of the shared task, emotion detection in code-switching text, in NLPCC 2018. The submitted systems are expected to automatically determine the emotions in the Chinese-English code-switching text. Different from monolingual text, code-switching text contains more than one language, and the emotion can be expressed by either monolingual or bilingual form. Hence, the challenges are: how to integrate both monolingual and bilingual forms to detect emotion, and how to bridge the gap to between two languages. Our shared task has 19 team participants. The highest F-score was 0.515. In this paper, we introduce the task, the corpus, the participating teams, and the evaluation results.
KeywordsEmotion detection Code-switching text Annotation and evaluation
We would like to thank the participants for their valuable feedback and results. We should thank Dr. Sophia Yat Mei Lee and Helena Yan Ping Lau for their excellent works on corpus annotation and analysis. The work is supported by the National Natural Science Foundation of China (61331011, 61751206), and the Early Career Scheme (ECS) sponsored by the Research Grants Council of Hong Kong (No. PolyU 5593/13H).
- 1.Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? Sentiment classification using machine learning techniques. In: Empirical Methods in Natural Language Processing, pp. 79–86 (2002)Google Scholar
- 2.Yat, S., Lee, M., Li, S., Huang, C.: Annotating events in an emotion corpus. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation, pp. 3511–3516 (2014)Google Scholar
- 3.Solorio, T., Liu, Y.: Learning to predict code-switching points. In: Conference on Empirical Methods in Natural Language Processing, pp. 973–981(2008)Google Scholar
- 4.Adel, H., Vu, N., Schultz, T.: Combination of recurrent neural networks and factored language models for code-switching language modeling. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pp. 206–211 (2013)Google Scholar
- 6.Crossley, S.A., Kyle, K., McNamara, D.S.: Sentiment analysis and social cognition engine (SEANCE): an automatic tool for sentiment, social cognition, and social-order analysis. Behav. Res. Methods 49, 1–19 (2016)Google Scholar
- 7.Gupta U., Chatterjee A., Srikanth R., Agrawal P.: A sentiment-and-semantics-based approach for emotion detection in textual conversations (2017)Google Scholar
- 9.Jamatia, A., Das, A.: Part-of-speech tagging for code-mixed English-Hindi Twitter and Facebook chat messages. In: Recent Advances in Natural Language Processing, pp. 139–248 (2015)Google Scholar
- 10.Lee, S., Wang, Z.: Emotion in code-switching texts: corpus construction and analysis. In: Eighth Sighan Workshop on Chinese Language Processing, pp. 91–99 (2015)Google Scholar
- 11.Wang, Z., Zhang, Y., Lee, S., Li, S., Zhou, G.: A bilingual attention network for code-switched emotion prediction. In: Proceeding of COLING (2016)Google Scholar
- 12.Wang, Z., Lee, S., Li, S., Zhou, G.: Emotion analysis in code-switching text with joint factor graph model. In: Transactions on Audio Speech and Language Processing, pp. 469–480 (2017)Google Scholar