Abstract
This study aimed to develop an automated computer scoring system (ACSS) incorporating a Korean spell checker to assess students’ constructed responses and to check the efficacy of this system. To accomplish this, we examined the performance of automatic spelling correction in reporting and correcting spelling errors, the interaction of gender and grade level in making spelling errors, the relationship between spelling errors and academic achievement, and the scoring efficacy of an ACSS that incorporated a spell checker. The analysis of percentage, two-way ANOVA, t-test, Pearson’s correlation, and human–computer correspondence were conducted. The results revealed that an automatic spelling correction system could report 66.44% and correct 26.78% of all total misspelled words. We also found gender and grade-level differences in misspelling words. Students misspelled fewer words as they advanced in grade level, and male students misspelled more words than females. In terms of the relationship between spelling errors and concepts, we found that the number of concepts included in student’s responses had a significant relationship with the total number of written words and misspelled words. This indicates that students who made more spelling errors had discussed more concepts in their responses. Based on these results, we discuss practical implications for preventing students’ responses being scored lower due to spelling errors caused by being less attentive using an ACSS with a spelling correction system.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets generated during the current study are not publicly available but are available from the corresponding author on reasonable request.
References
Abu-Mostafa, Y. S. (2012). Machines that think for themselves. Scientific American, 307(1), 78–81.
Adams, A. M., & Simmons, F. R. (2019). Exploring individual and gender differences in early writing performance. Reading and Writing, 32(2), 235–263.
Adams, A., Simmons, F., & Willis, C. (2015). Exploring relationships between working memory and writing: Individual differences associated with gender. Learning and Individual Differences, 40, 101–107.
Ahn, E.-J., & Kim, J.-M. (2010). The expository writing abilities of school-aged children. Communication Sciences and Disorders, 15(3), 321–336.
Al-Oudat, A. (2017). spelling errors in english writing committed by english-major students at BAU. Journal of Literature, Languages and Linguistics, 32, 43–47.
Alsaawi, A. (2015). Spelling errors made by Arab learners of English. International Journal of Linguistics, 7(5), 55–67.
Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater® V. 2. The Journal of Technology, Learning and Assessment, 4(3).
Babayiğit, S. (2015). The dimensions of written expression: Language group and gender differences. Learning and Instruction, 35, 33–41.
Bae, H. S. (2015). Orthographic, morphological, and syntactic development of school aged children depending on sentence completion writing. Korean Journal of Linguistics, 40(3), 403–421.
Bahr, R. H., Silliman, E. R., Berninger, V. W., & Dow, M. (2012). Linguistic pattern analysis of misspellings of typically developing writers in grades 1–9. Journal of Speech, Language, and Hearing Research, 55, 1587–1599.
Beggrow, E. P., Ha, M., Nehm, R. H., Pearl, D., & Boone, W. J. (2014). Assessing scientific practices using machine-learning methods: How closely do they match clinical interview performance? Journal of Science Education and Technology, 23(1), 160–182.
Boivin, M. C., & Pinsonneault, R. (2018). Errors in syntax, grammatical spelling and lexical spelling of Quebec students in the context of written production. Canadian Journal of Applied Linguistics, 21(1), 43–70.
Bourassa, D. C., & Treiman, R. (2001). Spelling development and disability: The importance of linguistic factors. Language, Speech, and Hearing Services in Schools, 32(3), 172–181.
Bridgeman, B., Trapani, C., & Attali, Y. (2012). Comparison of human and machine scoring of essays: Differences by gender, ethnicity, and country. Applied Measurement in Education, 25(1), 27–40.
Campbell, M. L. (2015). Multiple-Choice exams and guessing: Results from a one-year study of general chemistry tests designed to discourage guessing. Journal of Chemical Education, 92(7), 1194–1200.
Castro, S. L., & Limpo, T. (2018). Examining potential sources of gender differences in writing: The role of handwriting fluency and self-efficacy beliefs. Written Communication, 35(4), 448–473.
Cordeiro, C., Castro, S. L., & Limpo, T. (2018). Examining potential sources of gender differences in writing: The role of handwriting fluency and self-efficacy beliefs. Written Communication, 35(4), 448–473.
Dodd, B., & Carr, A. (2003). Young children’s letter-sound knowledge. Language, Speech, and Hearing Services in Schools, 34(2), 128–137.
Donovan, M. S., & Bransford, J. D. (Eds.). (2005). How students learn: History, mathematics and science in the classroom. The National Academies Press.
Earl, L. M. (2013). Assessment for learning; Assessment as learning: Changing practices means changing beliefs. Assessment, 80, 63–71.
Edwards, F. (2013). Quality assessment by science teachers: Five focus areas. Science Education International, 24(2), 212–226.
Edwards, J. H., & Liu, J. (2018). Peer response in second language writing classrooms. University of Michigan Press.
Evmenova, A. S., Graff, H. J., Jerome, M. K., & Behrmann, M. M. (2010). Word prediction programs with phonetic spelling support: Performance comparisons and impact on journal writing for students with writing difficulties. Learning Disabilities Research & Practice, 25(4), 170–182.
Fink-Chorzempa, B., Graham, S., & Harris, K. R. (2005). What can I do to help young children who struggle with writing? Teaching Exceptional Children, 37(5), 64–66.
Flor, M., & Futagi, Y. (2013). Producing an annotated corpus with automatic spelling correction. In S. Granger, G. Gilquin, & F. Meunier (Eds.), Twenty years of learner corpus research: Looking back, moving ahead (pp. 139–154). Presses universitaires de Louvain.
Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387.
Foorman, B. R., & Petscher, Y. (2010). Development of spelling and differential relations to text reading in grades 3–12. Assessment for Effective Intervention, 36(1), 7–20.
Frey, N., & Fisher, D. (2011). The formative assessment action plan: Practical steps to more successful teaching and learning. ASCD.
Ga, E.-A. (2010). A study on development of the writing ability: Focused on the expository wiring for 6th to 10th graders. Journal of CheongRam Korean Language Education, 41, 139–168.
Gelati, C. (2012). Female superiority and gender similarity effects and interest factors in writing. In V. W. Berninger (Ed.), Past, present, and future contributions of cognitive writing research to cognitive psychology. Psychology Press.
Graham, S., Harris, K. R., & Hebert, M. (2011). It is more than just the message: Presentation effects in scoring writing. Focus on Exceptional Children, 44(4), 1–12.
Ha, M., & Nehm, R. H. (2016). The impact of misspelled words on automated computer scoring: A case study of scientific explanations. Journal of Science Education and Technology, 25(3), 358–374.
Ha, M., Nehm, R. H., Urban-Lurain, M., & Merrill, J. E. (2011). Applying computerized-scoring models of written biological explanations across courses and colleges: Prospects and limitations. CBE—Life Sciences Education, 10(4), 379–393.
Haudek, K. C., Prevost, L. B., Moscarella, R. A., Merrill, J., & Urban-Lurain, M. (2012). What are they thinking? Automated analysis of student writing about acid-base chemistry in introductory biology. CBE—Life Sciences Education, 11(3), 283–293.
Hayes, J., & Flower, L. (1980). Identifying the organization of writing processes. In L. Gregg & E. Steinberg (Eds.), Cognitive processes in writing: An interdisciplinary approach (pp. 3–30). Lawrence Erlbaum.
Heitink, M. C., Van der Kleij, F. M., Veldkamp, B. P., Schildkamp, K., & Kippers, W. B. (2016). A systematic review of prerequisites for implementing assessment for learning in classroom practice. Educational Research Review, 17, 50–62.
Horne, J. (2007). Gender differences in computerised and conventional educational tests. Journal of Computer Assisted Learning, 23(1), 47–55.
Hull, D. (1993). Using statistical testing in the evaluation of retrieval experiments. In R. Korfhage, E. Rasmussen, and P. Willett (Eds.), SIGIR ’93 Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 329–338). New York, USA: ACM.
Ifenthaler, D., Adcock, A. B., Erlandson, B. E., Gosper, M., Greiff, S., & Pirnay-Dummer, P. (2014). Challenges for education in a connected world: Digital learning, data rich environments, and computer-based assessment—Introduction to the inaugural special issue of technology, knowledge and learning. Technology, Knowledge and Learning, 19(1–2), 121–126.
Im, C. (2017). Comparison and analysis of wiring development of middle and senior grade students in elementary school. Journal of Learner-Centered Curriculum and Instruction, 17(15), 485–508.
Jeong, S. S., Choi, Y. H., Han, K. T., Suh, C. D., and Lim, I. C. (1996). Morphological analysis by one pass parsing on Korean sentences include spacing faults. In Proceedings of conference of the institute of electronics and information engineers, (pp. 229–232).
Jordan, S., & Mitchell, T. (2009). e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology, 40(2), 371–385.
Kang, S., Shim, K., Lee, H., Lim, H., & Yoon, B. (1995). A study on morphological analysis for spoken language. Electronics and Telecommunications Research Institute.
Kim, R.-J. (2015). The establishment of word and the concept of word-phrase. Korean Language and Literature, 171, 5–39.
Kim, Y. S., Al Otaiba, S., Wanzek, J., & Gatlin, B. (2015). Toward an understanding of dimensions, predictors, and the gender gap in written composition. Journal of Educational Psychology, 107(1), 79–95.
Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405.
Lee, K. I., & Ahn, T. S. (2003). Development of POS Tagging System Independent to Word Spacing. In Annual Conference on Human and Language Technology. Human and Language Technology.
Lee, D., Yeon, J., Hwang, I., & Lee, S. (2010). KKMA: A tool for utilizing Sejong corpus based on relational database. Journal of KIISE: Computing Practices and Letters, 16(11), 1046–1050.
Lee, J. H., Kim, M., & Kwon, H. C. (2020). Comparison of context-sensitive spelling error correction using embedding techniques. Journal of Korean Institute of Information Scientists and Engineers, 47(2), 147–154.
Lee, S.-U. (2003). Some problems of word-spacing in Korean. The Korean Language and Literature, 134, 123–153.
Limbrick, L., Wheldall, K., & Madelaine, A. (2011). Why do more boys than girls have a reading disability? A review of the evidence. Australasian Journal of Special Education, 35, 1–24. https://doi.org/10.1375/ajse.35.1.1
Lukhele, R., Thissen, D., & Wainer, H. (1994). On the relative value of multiple-choice, constructed response, and examinee-selected items on two achievement tests. Journal of Educational Measurement, 31(3), 234–250.
Lunsford, A. A., & Lunsford, K. J. (2008). Mistakes are a fact of life: A national comparative study. College Composition and Communication, 59(4), 781–806.
MacArthur, C. A., Graham, S., & Fitzgerald, J. (Eds.). (2008). Handbook of writing research. Guilford Press.
Malecki, C. K., & Jewell, J. (2003). Developmental, gender, and practical considerations in scoring curriculum-based measurement writing probes. Psychology in the Schools, 40(4), 379–390.
Manning, C. D., Raghavan, P., & Schutze, H. (2008). Introduction to information retrieval. Cambridge University Press.
Mao, L., Liu, O. L., Roohr, K., Belur, V., Mulholland, M., Lee, H. S., & Pallant, A. (2018). Validation of automated scoring for a formative assessment that employs scientific argumentation. Educational Assessment, 23(2), 121–138.
Martin-Lacroux, C. (2017). “Without the spelling errors I would have shortlisted her…”: The impact of spelling errors on recruiters’ choice during the personnel selection process. International Journal of Selection and Assessment, 25(3), 276–283.
McKenna, P. (2019). Multiple choice questions: Answering correctly and knowing the answer. Interactive Technology and Smart Education., 16(1), 59–73.
Midgette, E., Haria, P., & MacArthur, C. (2008). The effects of content and audience awareness goals for revision on the persuasive essays of fifth-and eighth-grade students. Reading and Writing, 21(1–2), 131–151.
Mizuno, K., Tanaka, M., Fukuda, S., Sasabe, T., Imai-Matsumura, K., & Watanabe, Y. (2011). Changes in cognitive functions of students in the transitional period from elementary school to junior high school. Brain and Development, 33(5), 412–420.
Nagata, R., Whittaker, E., and Sheinman, V. (2011). Creating a manually error-tagged and shallow-parsed learner corpus. In D. Lin (Ed.). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, (pp. 1210–1219). Stroudsburg, USA: Association for Computational Linguistics.
Nassaji, H. (2007). The development of spelling and orthographic knowledge in English as an L2: A longitudinal case study. Canadian Journal of Applied Linguistics/revue Canadienne De Linguistique Appliquée, 10(1), 77–98.
Nehm, R. H., & Haertig, H. (2012). Human vs computer diagnosis of students’ natural selection knowledge: Testing the efficacy of text analytic software. Journal of Science Education and Technology, 21(1), 56–73.
Ocal, T., & Ehri, L. (2017). Spelling ability in college students predicted by decoding, print exposure, and vocabulary. Journal of College Reading and Learning, 47(1), 58–74.
Oh, H. J. (2008). A study on the language awareness change in the spacing words. 우리어문연구, 30, 383–406.
Park, T.-H., Kang, B.-R., Im, C.-T., & Lee, Y.-S. (2005). The investigation on quality and development related with the national language expression in writing of the elementary students. Korean Language Education Research, 23, 273–299.
Protopapas, A., Fakou, A., Drakopoulou, S., Skaloumbakas, C., & Mouzaki, A. (2013). What do spelling errors tell us? Classification and analysis of errors made by Greek schoolchildren with and without dyslexia. Reading and Writing, 26(5), 615–646.
Puteh, S. N., Rahamat, R., & Karim, A. A. (2010). Writing in the second language: Support and help needed by the low achievers. Procedia-Social and Behavioral Sciences, 7, 580–587.
Reece, C., & Treiman, R. (2001). Children’s spelling of syllabic/r/and letter-name vowels: Broadening the study of spelling development. Applied Psycholinguistics, 22(2), 139–165.
Reilly, D., Neumann, D. L., & Andrews, G. (2019). Gender differences in reading and writing achievement: Evidence from the National Assessment of Educational Progress (NAEP). American Psychologist, 74(4), 445–458.
Reynolds, M. R., Scheiber, C., Hajovsky, D. B., Schwartz, B., & Kaufman, A. S. (2015a). Gender differences in academic achievement: Is writing an exception to the gender similarities hypothesis? The Journal of Genetic Psychology: Research and Theory on Human Development, 176(4), 211–234.
Reynolds, M. R., Scheiber, C., Hajovsky, D. B., Schwartz, B., & Kaufman, A. S. (2015b). Gender differences in academic achievement: Is writing an exception to the gender similarities hypothesis? The Journal of Genetic Psychology, 176(4), 211–234.
Russell, M., and Wei, T. (2004). The influence of computer-print on rater scores. Practical Assessment, Research and Evaluation, 9(10).
Satoshi, T. (2017). First learned artificial intelligence: Basics of artificial intelligence algorithms and infrastructure for developers (K. Song, Trans.). Seoul: Hanbit Media. (Original work published 2016).
Shermis, M. D., & Burstein, J. C. (2003). Automated essay scoring: A cross-disciplinary perspective. Lawrence Erlbaum Associates.
Shim, K. (2015). Automatic word spacing using raw corpus and a morphological analyzer. Journal of KIISE: Computing Practices and Letters, 42(1), 68–75.
Shin, H. C. (2000). A study of word spacing using of morphological analysis. Korean Linguistics, 12, 167–185.
Shin, H. S. (2008). Developmental differences in argumentative writing performance by analytic assessment. Korean Journal of Educational Research, 46(1), 1–29.
Su, L. T. (1994). The relevance of recall and precision in user evaluation. Journal of the American Society for Information Science, 45(3), 207–217.
Sukkarieh J., Bolge E. (2008). Leveraging C-Rater’s automated scoring capability for providing instructional feedback for short constructed responses. In: Woolf B.P., Aïmeur E., Nkambou R., Lajoie S. (Eds.), Lecture Notes in Computer Science: Vol. 5091. Intelligent Tutoring Systems (pp. 779–783).
Talwar, A., Cote, N. G., & Binder, K. S. (2014). Investigating predictors of spelling ability for adults with low literacy skills. Journal of Research and Practice for Adult Literacy, Secondary, and Basic Education, 3(2), 35–50.
Tumlin, J., & Heller, K. W. (2004). Using word prediction software to increase typing fluency with students with physical disabilities. Journal of Special Education Technology, 19(3), 5–14.
Wagner, R. K., Puranik, C. S., Foorman, B., Foster, E., Wilson, L. G., Tschinkel, E., & Kantor, P. T. (2011). Modeling the development of written language. Reading and Writing, 24(2), 203–220.
Wolfe, E. W., Song, T., & Jiao, H. (2016). Features of difficult-to-score essays. Assessing Writing, 27, 1–10.
Acknowledgements
This work was supported by the Korea Foundation for the Advancement of Science & Creativity (KOFAC), and funded by the Korean Government (MOE).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lee, H., Ha, M., Lee, J. et al. Spelling Errors in Korean Students’ Constructed Responses and the Efficacy of Automatic Spelling Correction on Automated Computer Scoring. Tech Know Learn 28, 185–205 (2023). https://doi.org/10.1007/s10758-021-09568-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10758-021-09568-5