Abstract
Multiple approaches have been proposed for the automated classification of argumentative components. However, few studies have focused on argumentation in students’ essays and how automated classification can support the development of automated writing evaluation (AWE) systems in intelligent tutoring systems. In this study, linguistics features (related to positionality, semantic similarity, part-of-speech tags, named entity, and syntactic dependency) were obtained from 314 essays written by first-year college students. These features were used to build an algorithm to classify 2264 argumentative components found in the essays into four categories (final claim, primary claim, data, and other). Results indicated a Random Forest model (using five repeats of 10-fold cross-validation) achieved an overall F1-score of 0.78 in the classification and that the positionality, semantic similarity, and syntactic dependency features played the most critical roles. The algorithm can help inform the development of AWE algorithms to drive feedback on argumentative essays and help developing writers improve their argumentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aker, A.,et al.: What works and what does not: classifier and feature analysis for argument mining. In: Proceedings of the 4th Workshop on Argument Mining. pp. 91–96 (2017)
Al Khatib, K., Wachsmuth, H., Hagen, M., Köhler, J., Stein, B.: Cross-domain mining of argumentative text through distant supervision. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1395–1404 (2016)
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and regression trees, vol. 432, pp. 151–166. Wadsworth. International Group, Belmont (1984)
Burstein, J., et al.: Computer analysis of essay content for automated score prediction: A prototype automated scoring system for GMAT analytical writing assessment essays. ETS Res. Rep. Ser. 1998, i–67 (1998)
Cer, D., et al.: Universal sentence encoder. arXiv preprint arXiv:1803.11175 (2018)
Fleiss, J.L., Levin, B., Paik, M.C., et al.: The measurement of interrater agreement. Stat. Methods Rates Proport. 2, 22–23 (1981)
Habernal, I., Gurevych, I.: Exploiting debate portals for semi-supervised argumentation mining in user-generated web discourse. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pp. 2127–2137 (2015)
Habernal, I., Eckle-Kohler, J., Gurevych, I.: Argumentation Mining on the Web from Information Seeking Perspective. In: ArgNLP (2014)
Hirvela, A.: Preparing English language learners for argumentative writing. L2 writing in secondary classrooms: Student experiences, academic issues, and teacher education, pp, 67–86 (2013)
Honnibal, M., Montani, I.: spaCy 2: natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. 7, 411–420 (2017)
Klebanov, B.B., Stab, C., Burstein, J., Song, Y., Gyawali, B., Gurevych, I.: Argumentation: Content, structure, and relationship with essay quality. In: Proceedings of the Third Workshop on Argument Mining (ArgMining2016), pp. 70–75 (2016)
Knott, A., Dale, R.: Using linguistic phenomena to motivate a set of coherence relations. Discourse Process. 18, 35–62 (1994)
Lawrence, J., Reed, C.: Argument mining: a survey. Comput. Linguist. 45, 765–818 (2020)
Marcu, D., Echihabi, A.: An unsupervised approach to recognizing discourse relations. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 368–375 (2002)
McNamara, D.S., et al.: The Writing-Pal: Natural language algorithms to support intelligent tutoring on writing strategies. In: K-12 Education: Concepts, Methodologies, Tools, and Applications, pp. 780–793. IGI Global (2014)
Merity, S., Murphy, T., Curran, J.R.: Accurate argumentative zoning with maximum entropy models. In: Proceedings of the 2009 Workshop on Text and Citation Analysis for Scholarly Digital Libraries (NLPIR4DL), pp. 19–26 (2009)
Mochales, R., Moens, M.-F.: Argumentation mining. Artif. Intell. Law. 19, 1–22 (2011)
Nussbaum, E.M., Kardash, C.M., Graham, S.E.: The effects of goal instructions and text on the generation of counterarguments during writing. J. Educ. Psychol. 97, 157 (2005)
Ong, N., Litman, D., Brusilovsky, A.: Ontology-based argument mining and automatic essay scoring. In: Proceedings of the First Workshop on Argumentation Mining, pp. 24–28 (2014)
Palau, R.M., Moens, M.-F.: Argumentation mining: the detection, classification and structure of arguments in text. In: Proceedings of the 12th International Conference on Artificial Intelligence and Law, pp. 98–107 (2009)
Pedregosa, F., et al.: Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Qin, J., Karabacak, E.: The analysis of Toulmin elements in Chinese EFL university argumentative writing. System 38, 444–456 (2010)
Stab, C., Gurevych, I.: Identifying argumentative discourse structures in persuasive essays. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 46–56 (2014)
Stab, C., Gurevych, I.: Parsing argumentation structures in persuasive essays. Comput. Linguist. 43, 619–659 (2017)
Stapleton, P., Wu, Y.A.: Assessing the quality of arguments in students’ persuasive writing: a case study analyzing the relationship between surface structure and substance. J. Engl. Acad. Purp. 17, 12–23 (2015)
Toulmin, S.E.: The Uses of Argument. Cambridge University Press, Cambridge (1958)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wan, Q., Crossley, S., Tian, Y. (2022). Automated Classification of Argumentative Components in Students’ Essays. In: Crossley, S., Popescu, E. (eds) Intelligent Tutoring Systems. ITS 2022. Lecture Notes in Computer Science, vol 13284. Springer, Cham. https://doi.org/10.1007/978-3-031-09680-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-09680-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09679-2
Online ISBN: 978-3-031-09680-8
eBook Packages: Computer ScienceComputer Science (R0)