Skip to main content

Writing flexibility in argumentative essays: a multidimensional analysis

Reading and Writing volume 32, pages 1607–1634 (2019)Cite this article

Abstract

The assessment of argumentative writing generally includes analyses of the specific linguistic and rhetorical features contained in the individual essays produced by students. However, researchers have recently proposed that an individual’s ability to flexibly adapt the linguistic properties of their writing may more accurately capture their proficiency. However, the features of the task, learner, and educational context that influence this flexibility remain largely unknown. The current study extends this research by examining relations between linguistic flexibility, reading comprehension ability, and feedback in the context of an automated writing evaluation system. Students (n = 131) wrote and revised six argumentative essays in an automated writing evaluation system and were provided both summative and formative feedback on their writing. Additionally, half of the students had access to a spelling and grammar checker that provided lower-level feedback during the writing period. The results provide evidence for the supposition that skilled writers demonstrate linguistic flexibility across the argumentative essays that they produce. However, analyses also indicate that lower-level feedback (i.e., spelling and grammar feedback) have little to no impact on the properties of students’ essays nor on their variability across prompts or drafts. Overall, the current study provides important insights into the role of flexibility in argumentative writing skill and develops a strong foundation on which to conduct future research and educational interventions.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

References

  • Allen, L. K., Crossley, S. A., Snow, E. L., & McNamara, D. S. (2014). Game-based writing strategy tutoring for second language learners: Game enjoyment as a key to engagement. Language Learning and Technology, 18, 124–150.

    Google Scholar 

  • Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2016). Computer-based writing instruction. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (2nd ed., pp. 316–329). New York, NY: Guilford Press.

    Google Scholar 

  • Allen, L. K., & Perret, C. A. (2016). Commercialized writing systems. In D. S. McNamara & S. A. Crossley (Eds.), Adaptive educational technologies for literacy instruction (pp. 145–162). NY: Taylor & Francis, Routledge.

    Chapter  Google Scholar 

  • Allen, L. K., Snow, E. L., & McNamara, D. S. (2016). The narrative waltz: The role of flexibility on writing performance. Journal of Educational Psychology, 108, 911–924.

    Article  Google Scholar 

  • Allen, L. K., Snow, E. L., & McNamara, D. S. (2014). The long and winding road: Investigating the differential writing patterns of high and low skilled writers. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th international conference on educational data mining (pp. 304–307). London

  • Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater vol 2. Journal of Technology, Learning, and Assessment, 4(3), 3.

    Google Scholar 

  • Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1–48.

    Article  Google Scholar 

  • Biancarosa, G., & Snow, C. E. (2006). Reading next: A vision for action and research in middle and high school literacy—A report from the Carnegie Corporation of New York (2nd ed.). Washington, DC: Alliance for Excellent Education.

    Google Scholar 

  • Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37, 639–668.

    Article  Google Scholar 

  • Crossley, S. A., Roscoe, R. D., & McNamara, D. S. (2014). What is successful writing? An investigation into the multiple ways writers can write high quality essays. Written Communication, 31, 181–214.

    Article  Google Scholar 

  • Crossley, S. A., Weston, J., McLain-Sullivan, S. T., & McNamara, D. S. (2011). The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication, 28, 282–311.

    Article  Google Scholar 

  • Crossley, S. A., Kyle, K., Allen, L. K., & McNamara, D. S. (2014). The importance of grammar and mechanics in writing assessment and instruction: Evidence from data mining. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th international conference on educational data mining (pp. 300–303). London

  • Crossley, S. A., Varner (Allen), L. K., & McNamara, D. S. (2013). Cohesion-based prompt effects in argumentative writing. In C. Boonthum-Denecke & G. M. Youngblood (Eds.), Proceedings of the 26th annual flordia artificial intelligence research society (FLAIRS) conference (pp. 202–207). Menlo Park, CA: The AAAI Press.

  • Deane, P. (2013). On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing, 18, 7–24.

    Article  Google Scholar 

  • Dikli, S. (2006). An overview of automated scoring of essays. Journal of Technology, Learning, and Assessment, 5, 3–35.

    Google Scholar 

  • Duran, N., Bellissens, C., Taylor, R., & McNamara, D. (2007). In D. S. McNamara & G. Trafton (Eds.), Qualifying text difficulty with automated indices of cohesion and semantics (pp. 233–238). Austin, TX: Cognitive Science Society.

    Google Scholar 

  • Everitt, B. (1998). The Cambridge dictionary of statistics. Cambridge, NY: Cambridge University Press.

    Google Scholar 

  • Ferretti, R., & Fan, Y. (2016). Argumentative writing. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (2nd ed., pp. 301–315). NY: Guilford.

    Google Scholar 

  • Flower, L. S., & Hayes, J. (1981). A cognitive process theory of writing. College Composition and Communication, 32, 365–387.

    Article  Google Scholar 

  • Graesser, A. C., & McNamara, D. S. (2011). Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science, 2, 371–398.

    Article  Google Scholar 

  • Graesser, A. C., McNamara, D. S., & Kulikowich, J. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40, 223–234.

    Article  Google Scholar 

  • Graham, S., Hebert, M., & Harris, K. R. (2015). Formative assessment and writing: A meta-analysis. Elementary School Journal, 115, 524–547.

    Google Scholar 

  • Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools—A report to Carnegie Corporation of New York. Washington, DC: Alliance for Excellent Education.

    Google Scholar 

  • Guo, L., Crossley, S. A., & McNamara, D. S. (2013). Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, 18, 218–238.

    Article  Google Scholar 

  • Haswell, R. H. (2000). Documenting improvement in college writing: A longitudinal approach. Written Communication, 17(3), 307–352.

    Article  Google Scholar 

  • Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing. In C. M. Levy & L. S. Ransdell (Eds.), The science of writing: Theories, methods, individual differences and applications (pp. 1–27). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Kellogg, R. T. (2008). Training writing skills: A cognitive developmental perspective. Journal of Writing Research, 1, 1–26.

    Article  Google Scholar 

  • Kim, M., & Crossley, S. A. (2018). Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing, 37, 39–56.

    Article  Google Scholar 

  • Kim, Y. G., Schatschneider, C., Wanzek, J., Gatlin, B., & Otaiba, S. (2017). Writing evaluation: Rater and task effects on the reliability of writing scores for children in Grades 3 and 4. Reading and Writing, 30, 1287–1310.

    Article  Google Scholar 

  • MacGinitie, W. H., & MacGinitie, R. K. (1989). Gates MacGinitie reading tests. Chicago: Riverside.

    Google Scholar 

  • McNamara, D. S., & Allen, L. K. (2018). Toward an integrated perspective of writing as a discourse process. In M. Schober, A. Britt, & D. N. Rapp (Eds.), Handbook of discourse processes (2nd ed.). New York: Routledge.

    Google Scholar 

  • McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., & Dai, J. (2015). Natural language processing in a writing strategy tutoring system: Hierarchical classification approach to automated essay scoring. Assessing Writing, 23, 35–59.

    Article  Google Scholar 

  • McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • National Center for Education Statistics. (2012). The Nation’s Report Card: Writing 2011 (NCES 2012-470). Washington, DC: Institute for Education Sciences, U.S. Department of Education.

    Google Scholar 

  • National Commission on Writing. (2004). Writing: A ticket to work. Or a ticket out. New York: College Board.

    Google Scholar 

  • Perelman, L. (2012). Construct validity, length, score, and time in holistically graded writing assessments: The case against automated essay scoring (AES). In C. Bazerman, C. Dean, J. Early, K. Lunsford, S. Null, P. Rogers, & A. Stansell (Eds.), International advances in writing research: Cultures, places, measures (pp. 121–131). Fort Collins: Parlor Press.

    Google Scholar 

  • Phillips, L. M., Norris, S. P., Osmond, W. C., & Maynard, A. M. (2002). Relative reading achievement: A longitudinal study of 187 children from first through sixth grades. Journal of Educational Psychology, 94, 3–13.

    Article  Google Scholar 

  • Roscoe, R. D., Allen, L. K., Weston, J. L., Crossley, S. A., & McNamara, D. S. (2014). The Writing Pal intelligent tutoring system: Usability testing and development. Computers and Composition, 34, 39–59.

    Article  Google Scholar 

  • Roscoe, R. D., & McNamara, D. S. (2013). Writing pal: Feasibility of an intelligent writing strategy tutor in the high school classroom. Journal of Educational Psychology, 105, 1010–1025.

    Article  Google Scholar 

  • Roscoe, R. D., Varner, L. K., Crossley, S. A., & McNamara, D. S. (2013). Developing pedagogically-guided threshold algorithms for intelligent automated essay feedback. International Journal of Learning Technology, 8, 362–381.

    Article  Google Scholar 

  • Schoonen, R. (2005). Generaliability of writing scores: An application of structural equation modeling. Language Testing, 22, 1–30.

    Article  Google Scholar 

  • Shanahan, T. (1984). Nature of the reading-writing relation: An exploratory multivariate analysis. Journal of Educational Psychology, 76, 466–477.

    Article  Google Scholar 

  • Shanahan, T. (2016). Relationships between reading and writing development. In C. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (2nd ed., pp. 194–207). New York: Guilford.

    Google Scholar 

  • Shermis, M., & Burstein, J. (Eds.). (2003). Automated essay scoring: A cross-disciplinary perspective. Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Shermis, M. D., & Burstein, J. (Eds.). (2013). Handbook of automated essay evaluation: Current applications and future directions. New York: Routledge.

    Google Scholar 

  • Varner, L. K., Roscoe, R. D., & McNamara, D. S. (2013). Evaluative misalignment of 10th-grade student and teacher criteria for essay quality: An automated textual analysis. Journal of Writing Research, 5, 35–59.

    Article  Google Scholar 

  • Walton, D. N. (1992). Plausible argument in everyday conversation. Albany, NY: State University of New York Press.

    Google Scholar 

  • Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10, 1–24.

    Article  Google Scholar 

  • Weigle, S. C. (2013). English as a second language writing and automated essay evaluation. Handbook of automated essay evaluation: Current applications and new directions (pp. 36–54). New York: Routledge.

    Google Scholar 

  • Witte, S. P., & Faigley, L. (1981). Coherence, cohesion, and writing quality. College Composition and Communication, 32(2), 189–204.

    Article  Google Scholar 

  • Wong, B. (1999). Metacognition in writing. In R. Gallimore, L. P. Bernheimer, D. L. MacMillan, D. L. Speech, & S. Vaughn (Eds.), Developmental perspectives on children with high-incidence disabilities (pp. 183–198). Mahwah, NJ: Erlbaum.

    Google Scholar 

Download references

Acknowledgements

This research was supported in part by IES Grants R305A120707 and R305A180261 as well as the Office of Naval Research (Grant No. N00014-16-1-2611). Opinions, conclusions, or recommendations do not necessarily reflect the view of the Department of Education, IES, or the Office of Naval Research.

Author information

Authors and Affiliations

  1. Psychology Department, Mississippi State University, P.O. Box 6161, Mississippi State, MS, 39762, USA

    Laura K. Allen

  2. University of Nebraska, Biomechanics Research Building, 6001 Dodge Street, Omaha, NE, 68182-0202, USA

    Aaron D. Likens

  3. Learning Sciences Institute, Psychology Department, Arizona State University, P.O. Box 872111, Tempe, AZ, 85287-2111, USA

    Danielle S. McNamara

Authors
  1. Laura K. Allen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aaron D. Likens
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danielle S. McNamara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura K. Allen.

Appendix: Essay prompts

Appendix: Essay prompts

General Instructions

You will now have 25 min to write an essay on the prompt below.

The essay gives you an opportunity to show how effectively you can develop and express ideas. You should, therefore, take care to develop your point of view, present your ideas logically and clearly, and use language precisely.

Think carefully about the issue presented in the following excerpt and the assignment below.

[Prompt Specific Information]

Plan and write an essay in which you develop your point of view on this issue. Support your position with reasoning and examples taken from your reading, studies, experience, or observations.

Prompt 1

While some people promote competition as the only way to achieve success, others emphasize the power of cooperation. Intense rivalry at work or play or engaging in competition involving ideas or skills may indeed drive people either to avoid failure or to achieve important victories. In a complex world, however, cooperation is much more likely to produce significant, lasting accomplishments.

Do people achieve more success by cooperation or by competition?

Prompt 2

All around us appearances are mistaken for reality. Clever advertisements create favorable impressions but say little or nothing about the products they promote. In stores, colorful packages are often better than their contents. In the media, how certain entertainers, politicians, and other public figures appear is sometimes considered more important than their abilities. All too often, what we think we see becomes far more important than what really is.

Do images and impressions have a positive or negative effect on people?

Prompt 3

Loyalty is one of the essential attributes a person must have and must demand of others. Being loyal, faithful, or dedicated to someone or something, is not always easy. People often have conflicting loyalties, and there are no guidelines that help them decide to what or to whom they should be loyal. Moreover, people may be loyal to something harmful or bad.

Should people always maintain their loyalties, or is it sometimes necessary to switch sides?

Prompt 4

Many people believe that to move up the ladder of success and achievement, they must forget their past, repress it, and let it go. But others have just the opposite view. They see their old memories as a chance to reckon with their past and integrate past and present.

Do personal memories hinder or help people in their effort to learn from their past and succeed in the present?

Prompt 5

When we are young, we learn from parents and teachers that we should wait patiently for what we want. Few people would dispute the wisdom or truth of this teaching. Our society, however, with its mad rush and hurry and its insistence on instant gratification and quick responses, encourages and rewards impatience. Experience teaches us that we should not and do not have to wait.

Is it better for people to act quickly and expect quick responses from others rather than to wait patiently for what they want?

Prompt 6

From talent contests to the Olympics to the Nobel and Pulitzer prizes, we constantly seek to reward those who are “number one.” This emphasis on recognizing the winner creates the impression that other competitors, despite working hard and well, have lost. In many cases, however, the difference between the winner and the losers is slight. The wrong person may even be selected as the winner. Awards and prizes merely distract us from valuable qualities possessed by others besides the winners.

Do people place too much emphasis on winning?

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Allen, L.K., Likens, A.D. & McNamara, D.S. Writing flexibility in argumentative essays: a multidimensional analysis. Read Writ 32, 1607–1634 (2019). https://doi.org/10.1007/s11145-018-9921-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11145-018-9921-y

Keywords

  • Writing
  • Flexibility
  • Dynamics
  • Linguistics
  • Natural language processing
  • Individual differences
  • Intelligent tutoring systems
  • Feedback