Skip to main content
SpringerLink
Log in

Checking It Twice: Does Adding Spelling and Grammar Checkers Improve Essay Quality in an Automated Writing Tutor?

  • Conference paper
  • First Online:
Artificial Intelligence in Education (AIED 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11625))

Included in the following conference series:

Abstract

This study investigated the effect of incorporating spelling and grammar checking tools within an automated writing tutoring system, Writing Pal. High school students (n = 119) wrote and revised six persuasive essays. After initial drafts, all students received formative feedback about writing strategies. Half of the participants were also given access to spelling and grammar checking tools during the writing and revision periods. Linear mixed effects models revealed that essay quality for students in both conditions improved from initial draft to revision in terms of all aspects except essay unity. The availability of spelling and grammar checking yielded added improvements from initial draft to revision for several aspects of essay quality (i.e., conclusion, organization, voice, grammar/mechanics, and word choice), but other aspects were unaffected (i.e., introduction, body, unity, and sentence structure). The availability of spelling and grammar checking tools had no effect on holistic essay scores. These results indicate that automated spelling and grammar feedback contribute to modest, incremental improvements in writing quality that may complement automated strategy feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. National Center for Education Statistics: The Nation’s report Card: Washington, D.C.: Institute of Education Sciences, U.S. Department of Education (2012)

    Google Scholar 

  2. Kellogg, R.T., Raulerson III, B.A.: Improving the writing skills of college students. Psychon. Bull. Rev. 14(2), 237–242 (2007)

    Article  Google Scholar 

  3. Graham, S., Capizzi, A., Harris, K.R., Hebert, M., Morphy, P.: Teaching writing to middle school students: a national survey. Read. Writ. 27(6), 1015–1042 (2014)

    Article  Google Scholar 

  4. Kiuhara, S.A., Graham, S., Hawken, L.S.: Teaching writing to high school students: a national survey. J. Educ. Psychol. 101(1), 136–160 (2009)

    Article  Google Scholar 

  5. Allen, L.K., Jacovina, M.E., McNamara, D.S.: Computer-based writing instruction. In: MacArthur, C.A., Graham, S., Fitzgerald, J. (eds.) Handbook of Writing Research, 2nd edn, pp. 316–329. The Guilford Press, New York (2016)

    Google Scholar 

  6. Palermo, C., Thomson, M.M.: Teacher implementation of self-regulated strategy development with an automated writing evaluation system: effects on the argumentative writing performance of middle school students. Contemp. Educ. Psychol. 54, 255–270 (2018)

    Article  Google Scholar 

  7. Shermis, M.D., Burstein, J. (eds.): Handbook of Automated Essay Evaluation: Current Applications and New Directions. Routledge, New York (2013)

    Google Scholar 

  8. Stevenson, M., Phakiti, A.: The effects of computer-generated feedback on the quality of writing. Assess. Writ. 19, 51–56 (2014)

    Article  Google Scholar 

  9. Wilson, J., Czik, A.: Automated essay evaluation software in English language arts classrooms: effects on teacher feedback, student motivation, and writing quality. Comput. Educ. 100, 94–109 (2016)

    Article  Google Scholar 

  10. Crossley, S.A., Allen, L.K., McNamara, D.S.: The Writing Pal: a writing strategy tutor. In: Crossley, S.A., McNamara, D.S. (eds.) Adaptive educational technologies for literacy instruction, pp. 204–224. Routledge, New York (2016)

    Chapter  Google Scholar 

  11. Dai, J., Raine, R.B., Roscoe, R., Cai, Z., McNamara, D.S.: The writing-pal tutoring system: development and design. J. Eng. Comput. Innov. 2, 1–11 (2011)

    Google Scholar 

  12. McNamara, D.S., et al.: The writing-pal: natural language algorithms to support intelligent tutoring on writing strategies. In: McCarthy, P.M., Boonthum-Denecke, C. (eds.) Applied Natural Language Processing and Content Analysis: Identification, Investigation, and Resolution, pp. 298–311. IGI Global, Hershey (2012)

    Chapter  Google Scholar 

  13. Roscoe, R.D., McNamara, D.S.: Writing Pal: feasibility of an intelligent writing strategy tutor in the high school classroom. J. Educ. Psychol. 105, 1010–1025 (2013)

    Article  Google Scholar 

  14. McNamara, D.S., Crossley, S.A., Roscoe, R.D., Allen, L.K., Dai, J.: A hierarchical classification approach to automated essay scoring. Assess. Writ. 23, 35–59 (2015)

    Article  Google Scholar 

  15. Allen, L.K., Crossley, S.A., Snow, E.L., McNamara, D.S.: Game-based writing strategy tutoring for second language learners: game enjoyment as a key to engagement. Lang. Learn. Technol. 18, 124–150 (2014)

    Google Scholar 

  16. Roscoe, R.D., Allen, L.K., Weston, J.L., Crossley, S.A., McNamara, D.S.: The Writing Pal intelligent tutoring system: usability testing and development. Comput. Compos. 34, 39–59 (2014)

    Article  Google Scholar 

  17. Cutler, L., Graham, S.: Primary grade writing instruction: a national survey. J. Educ. Psychol. 100(4), 907–919 (2008)

    Article  Google Scholar 

  18. Morris, D., Blanton, L., Blanton, W.E., Perney, J.: Spelling instruction and achievement in six classrooms. Elem. Sch. J. 96(2), 145–162 (1995)

    Article  Google Scholar 

  19. Heift, T., Rimrott, A.: Learner responses to corrective feedback for spelling errors in CALL. System 36(2), 196–213 (2008)

    Article  Google Scholar 

  20. Graham, S., Perin, D.: A meta-analysis of writing instruction for adolescent students. J. Educ. Psychol. 99(3), 445–476 (2007)

    Article  Google Scholar 

  21. Crossley, S.A., Kyle, K., Allen, L.K., McNamara, D.S.: The importance of grammar and mechanics in writing assessment and instruction: evidence from data mining. In: Stamper, J., Pardos, Z., Mavrikis, M., McLaren,B.M. (eds.) Proceedings of the 7th International Conference on Educational Data Mining, London, UK, pp. 300–303 (2014)

    Google Scholar 

  22. Graham, S., Harris, K.R., Chambers, A.B.: Evidence-based practice and writing instruction. Handb. Writ. Res. 2, 211–226 (2016)

    Google Scholar 

  23. Graham, S., Harris, K.R.: Strategy instruction and the teaching of writing. Handb. Writ. Res. 5, 187–207 (2006)

    Google Scholar 

  24. Graham, S., Santangelo, T.: Does spelling instruction make students better spellers, readers, and writers? A meta-analytic review. Read. Writ. 27(9), 1703–1743 (2014)

    Article  Google Scholar 

  25. Morphy, P., Graham, S.: Word processing programs and weaker writers/readers: a meta-analysis of research findings. Read. Writ. 25(3), 641–678 (2012)

    Article  Google Scholar 

  26. Marshall, J.C.: Composition errors and essay examination grades re-examined. Am. Educ. Res. Assoc. 4, 375–385 (1967)

    Article  Google Scholar 

  27. Marshall, J.C., Powers, J.M.: Writing neatness, composition errors, and essay grades. J. Educ. Meas. 6, 97–101 (1969)

    Article  Google Scholar 

  28. Figuredo, L., Varnhagen, C.K.: Didn’t you run the spell checker? Effects of type of spelling error and use of a spell checker on perceptions of the authors. Read. Psychol. 26, 441–458 (2005)

    Article  Google Scholar 

  29. Boland, J.E., Queen, R.: If you’re house is still available, send me an email: personality influences reactions to written errors in email messages. PLoS ONE 11, e0149885 (2016)

    Article  Google Scholar 

  30. Johnson, A.C., Wilson, J., Roscoe, R.D.: College student perceptions of writing errors, text quality, and author characteristics. Assess. Writ. 34, 72–87 (2017)

    Article  Google Scholar 

  31. McCarthy, K.S., Likens, A.D., Johnson, A.M., Guerrero, T.A., McNamara, D.S.: Metacognitive overload!: Positive and negative effects of metacognitive prompts in an intelligent tutoring system. Int. J. Artif. Intell. Educ. 28(3), 1–19 (2018)

    Article  Google Scholar 

  32. Crawford, L., Lloyd, S., Knoth, K.: Analysis of student revisions on a state writing test. Assess. Eff. Interv. 33, 108–119 (2008)

    Google Scholar 

  33. Fitzgerald, J.: Research on revision in writing. Rev. Educ. Res. 57, 481–506 (1987)

    Article  Google Scholar 

  34. MacGinitie, W.H., MacGinitie, R.K., Cooter, R.B., Curry, S.: Assessment: Gates-MacGinitie reading tests. Read. Teach. 43(3), 256–258 (1989)

    Google Scholar 

  35. Miłkowski, M.: Developing an open-source, rule-based proofreading tool. Softw. Pract. Exp. 40(7), 543–566 (2010)

    Google Scholar 

  36. Naber, D.: A rule-based style and grammar checker. Master’s thesis, Universität Bielefeld (2003). http://www.danielnaber.de/publications

  37. Allen, L.K., Snow, E.L., Crossley, S.A., Jackson, G.T., McNamara, D.S.: Reading comprehension components and their relation to the writing process. L’année psychologique/Top. Cogn. Psychol. 114, 663–691 (2014)

    Article  Google Scholar 

  38. Bates, D., Maechler, M., Bolker, B., Walker, S.: lme4: linear mixed-effects models using Eigen and S4. R package version 1.1–7 (2015)

    Google Scholar 

  39. R Core Team: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2017)

    Google Scholar 

  40. Hughes, J., Team, R.C. Reghelper: helper functions for regression analysis. R package, version 0.3.3 (2017)

    Google Scholar 

  41. Strobl, C., et al.: Digital support for academic writing: a review of technologies and pedagogies. Comput. Educ. 131, 33–48 (2018)

    Article  Google Scholar 

  42. Attali, Y., Burstein, J.: Automated essay scoring with e‐rater® v. 2.0. ETS Research Report Series 2004. 2, i-21(2004)

    Google Scholar 

  43. Crossley, S.A., Roscoe, R.D., McNamara, D.S.: Using automatic scoring models to detect changes in student writing in an intelligent tutoring system. In: Proceedings of the 26th International Florida Artificial Intelligence Research Society (FLAIRS) Conference, pp. 208–213. The AAAI Press, Menlo Park (2013)

    Google Scholar 

  44. Roscoe, R.D., Snow, E.L., McNamara, D.S.: Feedback and revising in an intelligent tutoring system for writing strategies. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 259–268. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_27

    Chapter  Google Scholar 

  45. Roscoe, R.D., Snow, E.L., Allen, L.K., McNamara, D.S.: Automated detection of essay revising patterns: application for intelligent feedback in a writing tutor. Technol. Instr. Cogn. Learn. 10(1), 59–79 (2015)

    Google Scholar 

  46. Cavaleri, M.R., Dianati, S.: You want me to check your grammar again? The usefulness of an online grammar checker as perceived by students. Journal of Academic Language and Learning 10(1), A223–A236 (2016)

    Google Scholar 

  47. Potter, R., Fuller, D.: My new teaching partner? Using the grammar checker in writing instruction. Engl. J. 98(1), 36–41 (2008)

    Google Scholar 

  48. Allen, L.K., et al.: {ENTER}ing the time series {SPACE}: Uncovering the writing process through keystroke analysis. In: Proceedings of the 9th International Conference on Educational Data Mining (EDM 2016), pp. 22–29. International Educational Data Mining Society, Raleigh, NC (2016)

    Google Scholar 

  49. Likens, A.D., Allen, L.K., McNamara, D. S.: Keystroke dynamics predict essay quality. In: Gunzelmann, G., Howes, A., Tenbrink, T., Davelaar, E. (eds.) Proceedings of the 39th Annual Meeting of the Cognitive Science Society (CogSci 2017), pp. 2573–2578. Cognitive Science Society, London, UK (2017)

    Google Scholar 

Download references

Acknowledgements

This research was supported by grants from the U.S. DoEd Institute of Education Sciences (R305A120707 and R305A180261) and the U.S. DoD Office of Naval Research (N000141712300). Opinions, findings, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding sources.

Author information

Authors and Affiliations

  1. Georgia State University, Atlanta, GA, USA

    Kathryn S. McCarthy

  2. Arizona State University, Tempe, AZ, USA

    Rod D. Roscoe & Danielle S. McNamara

  3. University of Nebraska at Omaha, Omaha, NE, USA

    Aaron D. Likens

Authors
  1. Kathryn S. McCarthy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rod D. Roscoe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron D. Likens
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Danielle S. McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kathryn S. McCarthy .

Editor information

Editors and Affiliations

  1. University of Sao Paulo, Sao Paulo, Brazil

    Seiji Isotani

  2. University of Malaga, Málaga, Spain

    Eva Millán

  3. Carnegie Mellon University, Pittsburgh, PA, USA

    Amy Ogan

  4. DePaul University, Chicago, IL, USA

    Peter Hastings

  5. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce McLaren

  6. University College London, London, UK

    Rose Luckin

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

McCarthy, K.S., Roscoe, R.D., Likens, A.D., McNamara, D.S. (2019). Checking It Twice: Does Adding Spelling and Grammar Checkers Improve Essay Quality in an Automated Writing Tutor?. In: Isotani, S., Millán, E., Ogan, A., Hastings, P., McLaren, B., Luckin, R. (eds) Artificial Intelligence in Education. AIED 2019. Lecture Notes in Computer Science(), vol 11625. Springer, Cham. https://doi.org/10.1007/978-3-030-23204-7_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-23204-7_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-23203-0

  • Online ISBN: 978-3-030-23204-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation