Automated Chinese Essay Scoring Based on Multilevel Linguistic Features

  • Tao-Hsing ChangEmail author
  • Yao-Ting Sung
Part of the Chinese Language Learning Sciences book series (CLLS)


Writing assessments make up an important part of the learning process as one masters the important linguistic skill of writing. However, this process has not been implemented effectively or on a large scale because the task of essay scoring is very time-consuming. The solution to this problem is AES, where machines are used to automatically score essays. In fact, the application of AES to English learning has been successful. Due to differences in linguistic characteristics, a redesign is needed before AES can be applied to Chinese learning. The purpose of this chapter is to introduce ACES, an automated system for scoring Chinese essays, and explain the basic framework, design principles, and scoring accuracy of the system. Unlike some end-to-end AES systems, ACES’ basic framework is designed to provide more interpretative features. The experimental results show that the performance of the ACES system is stable and reliable, and on par with other commercial English AES systems.



This study was partially supported by the Ministry of Science and Technology, under the grant 107-2511-H-003 -022 -MY3; 104-2511-S-003 -018 -MY3; 107-2511-H-992-001-MY3; 104-2511-S-151-001-MY3, and the University Sprout Project―Chinese Language and Technology Center of National Taiwan Normal University, sponsored by the Ministry of Education, Taiwan.


  1. Attali, Y., & Burstein, J. (2006). Automated scoring with e-raterV.2. The Journal of Technology, Learning and Assessment, 4(3), 1–30.Google Scholar
  2. Bai, S. T., & Shi, A. W. (2002). A comparative study of figures of speech between Chinese and English. Journal of Xinzhou Teachers University, 18(1), 70–71.Google Scholar
  3. Burstein, J., Kukich, K., Wolff, S., Lu, C., Chodorow M., Braden-Harder, L., & Harris, M. D. (1998). Automated scoring using a hybrid feature identification technique. In Proceedings of the 36th Annual Meeting of the Association of Computational Linguistics (pp. 206–210). Montreal, Canada.Google Scholar
  4. Cai, J. G. (2006). Contrastive study of writing and rhetoric in English and Chinese. Shanghai, China: Fudan University Press.Google Scholar
  5. Chang. T. H. (2015). The development of Chinese word segmentation tool for educational text. In Proceedings of the 7th International Conference on Information (pp. 179–182). Taipei, Taiwan.Google Scholar
  6. Chang. T. H., Chen, H. C., & Yang, C. H. (2015). Introduction to a proofreading tool for Chinese spelling check task of SIGHAN-8. In Proceedings of the 8th SIGHAN Workshop on Chinese Language Processing (pp. 50–55). Beijing, China.Google Scholar
  7. Chang, T. H., & Lee, C. H. (2003). Automatic Chinese unknown word extraction using small-corpus-based method. In Proceedings of IEEE International Conference on Natural language processing and knowledge engineering (pp. 459–464). Beijing, China.Google Scholar
  8. Chang, T. H., & Lee, C. H. (2009). Automatic Chinese essay scoring using connections between concepts in paragraphs. In Proceedings of the International Conference on Asian Language Processing (pp. 265–268). Singapore.Google Scholar
  9. Chang, T. H., Lee, C. H., & Tam, H. P. (2007a). On issues of feature extraction in Chinese automatic essay scoring system. In Proceedings of the 13th International Conference on Artificial Intelligence in Education (pp. 545–547). Los Angeles, CA.Google Scholar
  10. Chang, T. H., Lee, C. H., & Tam, H. P. (2007b). On developing techniques for automated Chinese essay scoring: A case in ACES system. In Proceedings of the Forum for Educational Evaluation in East Asia (pp. 151–152). Taipei, Taiwan.Google Scholar
  11. Chang, T. H., Lee, C. H., Tsai, P. Y., & Tam, H. P. (2009). Automated essay scoring using set of literary sememes. Information: An International Interdisciplinary Journal, 12(2), 351–357.Google Scholar
  12. Chang, T. H., Liu, C. L., Su, S. Y., & Sung, Y. T. (2014). Integrating various features to grade students’ writings based on improved multivariate Bernoulli model. Information: An International Interdisciplinary Journal, 17(1), 45–52.Google Scholar
  13. Chen, K. J., Luo, C. C., Chang, M. C., Chen, F. Y., Chen, C. J., Huang, C. R., et al. (2003). Sinica Treebank. In A. Abeillé (Ed.), Treebanks: Building and using parsed corpora (pp. 231–248). Dordrecht: Springer.CrossRefGoogle Scholar
  14. Dong, Z., & Dong, Q. (2003). HowNet—A hybrid language and knowledge resource. In Proceedings of International Conference on Natural Language Processing and Knowledge Engineering (pp. 820–824). Beijing, China.Google Scholar
  15. Elliot, S. M. (2003). IntelliMetric: From here to validity. In M. D. Shermis & J. C. Burstein (Eds.), Automated essay scoring: A cross-disciplinary perspective (pp. 71–86). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  16. Jiao, C. Y. (2002). A syntactic comparison and transformation between English and Chinese. Journal of Yancheng Teachers College, 22(2), 83–87.Google Scholar
  17. Kaplan, R. B. (1966). Cultural thought patterns in intercultural education. Language Learning, 16(1–2), 1–20.CrossRefGoogle Scholar
  18. Lafferty, J., McCallum, A., & Pereira, F. C. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (pp. 282–289). Williamstown, MA.Google Scholar
  19. Lee, G. N. (1999). Contrastive studies of figures of speech in English and Chinese. Fuzhou, China: Fujian People’s Publishing House.Google Scholar
  20. Lee, X. L., & Zeng, K. (2001). Heterogeneity and homogeneity of sentence structure in English and Chinese. Journal of Shenyang University, 12(1), 52–55.Google Scholar
  21. Liu, L. J. (1999). A contrastive study of discourse structure in English and Chinese. Modern Foreign Languages, 86(4), 408–419.Google Scholar
  22. Manning, C. D., & Schütze, H. (1999). Foundations of statistical natural language processing. Cambridge, MA: MIT press.Google Scholar
  23. Peng, X., Ke, D., Chen, Z., & Xu, B. (2010). Automated Chinese essay scoring using vector space models. In Proceedings of the 4th International Universal Communication Symposium (pp. 149–153). Beijing, China.Google Scholar
  24. Ramineni, C., Trapani, C. S., Williamson, D. M., Davey, T., & Bridgeman, B. (2012). Evaluation of the e‐rater® scoring engine for the GRE® issue and argument prompts (ETS RR–12-02). Accessed August 31, 2017.
  25. Rudner, L. M., Garcia, V., & Welch, C. (2006). An evaluation of IntelliMetric™ essay scoring system. The Journal of Technology, Learning and Assessment, 4(4), 1–22.Google Scholar
  26. Rudner, L. M., & Liang, T. (2002). Automated essay scoring using Bayes’ theorem. The Journal of Technology, Learning, and Assessment, 1(2), 1–21.Google Scholar
  27. Scollon, R., Scollon, S. W., & Kirkpatrick, A. (2000). Contrastive discourse in Chinese and English: A critical appraisal. Beijing: Foreign Language Teaching and Research Press.Google Scholar
  28. Sung, Y. T., Lin, W. C., Dyson, S. B., Chang, K. E., & Chen, Y. C. (2015). Leveling L2 texts through readability: Combining multilevel linguistic features with the CEFR. The Modern Language Journal, 99(2), 371–391.CrossRefGoogle Scholar
  29. Sung, Y. T., Chang, T. H., Lin, W. C., Hsieh, K. S., & Chang, K. E. (2016). CRIE: An automated analyzer for Chinese texts. Behavior Research Methods, 48(4), 1238–1251.CrossRefGoogle Scholar
  30. Zeng, X. H. (1997). Enhancing English writing ability by comparing the difference of organization in paragraphs between English and Chinese. Journal of Nanchang Vocation-technical Teachers College, 4, 75–77.Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.National Kaohsiung University of Science and TechnologyKaohsiungTaiwan
  2. 2.National Taiwan Normal UniversityTaipeiTaiwan

Personalised recommendations