Abstract
Essay grading is an important issue in natural language processing. There are two challenges for Chinese essay grading, namely the subjectivity of expert grading standards and the lack of fine-grained labeled data. In this paper, we propose an automatic Chinese essay grading method based on multi-aspect expert knowledge. We introduce essay grading expert rules to turn the existing standards into indexes, such as ‘The Essay Grading Standards for College Entrance Examination’ and ‘The Chinese Curriculum Standards for Compulsory Education’. Based on the expert rules, we propose different encoders to learn multiple essay features in three aspects, namely the topic consistency, structure rationality and linguistics proficiency. An essay is graded by unifying the three grades in different aspects. Experimental results on two real datasets show the effectiveness of our method. We also analysis the influence of each aspect on the essay grading results. The experiment on the material essay grading dataset shows the practicability of our model in general exam scenarios.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ministry of Education the People’s Republic of China: Compulsory Education Chinese Curriculum Standard. People’s Education Press, Beijing (2011). ISBN: 9787303133178
General College Admissions Unified National Examination Outline in 2019. Higher Education Press, National Education Examination Authority (2018)
Page, E.B.: Grading essays by computer: progress report. In: Proceedings of the Invitational Conference on Testing Problems (1967)
Foltz, P.W., Laham, D., Landauer, T.K.: The intelligent essay assessor: applications to educational technology. Interact. Multim. Electr. J. Comput. Enhan. Learn. 1(2), 939–944 (1999)
Burstein, J.: The E-rater® scoring engine: automated essay scoring with natural language processing. In: Shermis, M.D., Burstein, J. (eds.) Automated Essay Scoring: A Cross-Disciplinary Perspective, pp. 113–121. Lawrence Erlbaum Associates, Mahwah (2003)
Rudner, L.M.: An evaluation of the IntelliMetric essay scoring system. J. Technol. Learn. Assess. 4(4), 3–21 (2006)
Rudner, L.M., Liang, T.: Automated essay scoring using Bayes’ theorem. J. Technol. Learn. Assess. 1(2), 1–22 (2002)
Larkey, L S.: A text categorization approach to automated essay grading. Automated Essay Scoring: A Cross-Disciplinary Perspective, 55–70 (2002)
Larkey L.S.: Automatic essay grading using text categorization techniques. In: 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 90–95. ACM, Melbourne (1998)
Dasgupta, T., Naskar, A., Dey, L., et al.: Augmenting textual qualitative features in deep convolution recurrent neural network for automatic essay scoring. In: 5th Meeting of the Association for Computational Linguistics, pp. 93–102. ACM, Melbourne (2018)
Dong, F., Zhang, Y., Yang, J.: Attention-based recurrent convolutional neural network for automatic essay scoring. In: 21st Conference on Computational Natural Language Learning, pp. 153–162. Association for Computational Linguistics, Vancouver (2017)
Wang, Y., Wei, Z., Zhou, Y., Huang, X.: Automatic essay scoring incorporating rating schema via reinforcement learning. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 791–797. Association for Computational Linguistics, Brussels (2018)
Tay, Y., Phan, M.C., Tuan, L.A., et al.: SkipFlow: incorporating neural coherence features for end-to-end automatic text scoring. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 5948–5955. AAAI Press, New Orleans (2018)
Alikaniotis, D., Yannakoudakis, H., Rei, M., et al.: Automatic text scoring using neural networks. In: the 54th Annual Meeting of the Association for Computational Linguistics, pp. 715–725. The Association for Computer Linguistics, Berlin (2016)
Jin, C., He, B., Hui, K., et al.: TDNN: a two-stage deep neural network for prompt-independent automated essay scoring. In: 56th Annual Meeting of the Association for Computational Linguistics, pp. 1088–1097. The Association for Computer Linguistics, Melbourne (2018)
Yifei, G.: Explainable essay grading method based on expert knowledge. Shandong University (2020)
Lu, X.: Automatic measurement of syntactic complexity in child language acquisition. Int. J. Corpus Linguist. 14(1), 3–28 (2009)
Na, H.: A corpus-based study on the syntactic features of primary school students’ compositions. Shanghai Normal University (2014)
Jäntschi, L., et al.: Pearson versus Spearman, Kendall’s Tau correlation analysis on structure-activity relationships of biologic active compounds (2005)
Hochreiter, S., Schmidhuber, J.: Long Short-Term Memory. Neural Comput. 9(8), 1735–1780 (1997)
Wang, Yequan, et al. “Attention-Based LSTM for Aspect-Level Sentiment Classification.” Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016, pp. 606–615
Kim Y.: Convolutional neural networks for sentence classification[C]. In: Empirical Methods in Natural Language Processing, pp. 1746–1751. The Association for Computer Linguistics, Doha (2014)
Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018
Cohen, J.: Weighted Kappa: Nominal Scale Agreement Provision for Scaled Disagreement or Partial Credit. Psychol. Bull. 70(4), 213–220 (1968)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, X., Xie, Y., Yang, T., Sun, Y. (2022). Understanding Expert Knowledge for Chinese Essay Grading. In: Sun, Y., et al. Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2021. Communications in Computer and Information Science, vol 1491. Springer, Singapore. https://doi.org/10.1007/978-981-19-4546-5_38
Download citation
DOI: https://doi.org/10.1007/978-981-19-4546-5_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-4545-8
Online ISBN: 978-981-19-4546-5
eBook Packages: Computer ScienceComputer Science (R0)
