Learning to compose digitally: the effect of prior computer use and keyboard activity on NAEP writing
Writing is critical for college and career readiness, yet secondary students in America are not good writers (National Center for Education Statistics, 2012). Unfortunately, researchers know relatively little about secondary students’ writing skills, and even less about their digital writing. In this study, we explored prior computer use, keyboard activity during writing, and their relations to writing achievement using the 8th grade 2011 National Assessment of Educational Progress writing assessment, the first national computer-based writing assessment for U.S. secondary students. We found that prior computer use predicted students’ writing skills directly (0.08) and indirectly (e.g., keypresses, 0.14) via keyboard activity during the test. We found differential effects for certain groups including current English learners and disadvantaged students. We also found a small positive interaction effect of prior use and keypresses on writing. That is, the benefits of prior computer use for school writing and the value of students’ additional keypresses on writing achievement were amplified when both were present.
KeywordsWriting NAEP Digital Middle school Literacy
This work has been supported by funding from the Spencer Foundation, Grant 201500153. This work is based on the 2011 National Assessment of Educational Progress in Writing Restricted-Use Data, available from the National Center for Education Statistics (NCES Pub. No. 2014476). https://nces.ed.gov/nationsreportcard/researchcenter/datatools.aspx.
- Acock, A. C. (2012). A gentle introduction to Stata (5th ed.). College Station, TX: Stata press.Google Scholar
- Applebee, A. N. (2011). Issues in large-scale writing assessment. Journal of Writing Assessment, 3(2), 81–98.Google Scholar
- Applebee, A. N., & Langer, J. A. (2011). A snapshot of writing instruction in middle schools and high schools. The English Journal, 100(6), 14–27.Google Scholar
- Bangert-Drowns, R. (1993). The word processor as an instructional tool: A meta-analysis of word processing in writing instruction. Review of Educational Research, 63(1), 69–93.Google Scholar
- Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.Google Scholar
- Bazerman, C. (2012). Writing, cognition and affect from the perspectives of sociocultural and historical studies of writing. In: Past, present and future contributions of cognitive writing research to cognitive psychology (pp. 89–104).Google Scholar
- Beaton, A., Rogers, A., Gonzalez, E., Hanly, M., Kolstad, A., Rust, K., et al. (2011). The NAEP Primer. Washington, DC: National Center for Educational Statistics.Google Scholar
- Berninger, V. W., Abbott, R. D., Augsburger, A., & Garcia, N. (2009). Comparison of pen and keyboard transcription modes in children with and without learning disabilities. Learning Disability Quarterly, 32(3), 123–141.Google Scholar
- Berninger, V. W., Whitaker, D., Feng, Y., Swanson, H. L., & Abbott, R. D. (1996). Assessment of planning, translating, and revising in junior high writers. Journal of School Psychology, 34(1), 23–52.Google Scholar
- Berninger, V. W., & Winn, W. D. (2006). Implications of advancements in brain research and technology for writing development, writing instruction, and educational evolution. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of Writing Research. NY: Guilford Press.Google Scholar
- Bolter, J. D. (1991). Writing space: The computer, hypertext, and the history of writing. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
- Bruce, B., Michaels, S., & Watson-Gegeo, K. (1985). How computers can change the writing process. Language Arts, 62(2), 143–149.Google Scholar
- Carnegie Council on Advancing Adolescent Literacy. (2010). Time to act: An agenda for advancing adolescent literacy for college and career success. New York, NY: Carnegie Corporation of New York. Retrieved from: https://www.carnegie.org/media/filer_public/8c/8d/8c8dfd82-b5fc-4bb9-8bd1-bb262175eaf4/ccny_report_2010_tta_agenda.pdf. Accessed 19 Feb 2019.
- Carr, P. (2017, Oct. 31). Statement on NAEP’s transition to digitally based assessments. https://nces.ed.gov/nationsreportcard/dba/statement.aspx. Accessed 19 Feb 2019.
- Costello, A. B., & Osborne, J. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation, 10(7). http://pareonline.net/getvn.asp?v=10&n=7. Accessed 19 Feb 2019.
- De La Paz, S., & Graham, S. (2002). Explicitly teaching strategies, skills & knowledge: Writing instruction in middle school classrooms. Journal of Education Psychology, 94(4), 687.Google Scholar
- Deane, P., Sabatini, J., & Fowles, M. (2012). Rethinking K-12 writing assessment to support best instructional practices. In International advances in writing research: Cultures, places, measures (pp. 83–101).Google Scholar
- DeVoss, D. N., Eidman-Aadahl, E., & Hicks, T. (2010). Because digital writing matters: Improving student writing in online and multimedia environments. New York: Wiley.Google Scholar
- Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College Composition and Communication, 32(4), 365–387.Google Scholar
- Goldberg, A., Russell, M., & Cook, A. (2003). The effect of computers on student writing: A meta-analysis of studies from 1992 to 2002. The Journal of Technology, Learning and Assessment, 2(1), 3–51.Google Scholar
- Graham, S., Bruch, J., Fitzgerald, J., Friedrich, L., Furgeson, J., Greene, K., et al. (2016). Teaching secondary students to write effectively (NCEE 2017-4002). Washington, DC: National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Education. Retrieved from the NCEE website: http://whatworks.ed.gov. Accessed 19 Feb 2019.
- Graham, S., & Hebert, M. (2010). Writing to read: Evidence for how writing can improve reading: A report from Carnegie Corporation of New York. New York: Carnegie Corporation of New York.Google Scholar
- Graham, S., Kiuhara, S., Harris, K., & Fishman, E. (2017). The relationship among strategic writing behavior, writing motivation, and writing performance with young, developing writers. The Elementary School Journal, 118(1), 82–104.Google Scholar
- Graham, S., McKeown, D., Kiuhara, S., & Harris, K. R. (2012). A meta-analysis of writing instruction for students in the elementary grades. Journal of Educational Psychology, 104(4), 879.Google Scholar
- Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99(3), 445–476.Google Scholar
- Johnson, E. G. (1992). The design of the national assessment of educational progress. Journal of Educational Measurement, 29(2), 95–110.Google Scholar
- Leu, D., Forzani, E., Rhoads, C., Maykel, C., Kennedy, C., & Timbrell, N. (2015). The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Research Quarterly, 50(1), 37–59.Google Scholar
- Little, T. (2013). Longitudinal structural equation modeling. New York: Guilford.Google Scholar
- McCutchen, D. (1996). A capacity theory of writing: Working memory in composition. Educational Psychology Review, 8, 299–325.Google Scholar
- McGrenere, J., & Ho, W. (2000). Affordances: Clarifying and evolving a concept. Paper presented at the Proceedings of Graphics Interface 2000, Montreal.Google Scholar
- Messick, S., Beaton, A., & Lord, F. (1983). National assessment of educational progress reconsidered: a new design for a new era.Google Scholar
- Morphy, P., & Graham, S. (2012). Word processing programs and weaker writers/readers: A meta-analysis of research findings. Reading and Writing, 25(3), 641–678.Google Scholar
- National Assessment Governing Board (NAGB). (2010). Writing framework for the 2011 National Assessment of Educational Progress. Washington, D.C.: U.S. Department of Education.Google Scholar
- National Center for Education Research and National Center for Special Education Research. (2017). Future directions for writing research at the secondary level. Technical Working Group Meeting Summary Report. Washington, DC: Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncer/whatsnew/techworkinggroup/pdf/WritingTWG.pdf. Accessed 19 Feb 2019.
- National Center for Education Statistics (NCES). (2009). The Nation’s Report Card: An overview of procedures for the NAEP Assessment. (NCES-2009-493). Washington, DC: Institute of Education Sciences, U.S. Department of Education.Google Scholar
- National Center for Education Statistics (NCES). (2012). The Nation’s Report Card: Writing 2011. (NCES 2012-4700) Institute of Education Sciences, Washington, DC: U.S. Department of Education. http://nces.ed.gov/nationsreportcard/pdf/main2011/2012470.pdf. Accessed 19 Feb 2019.
- National Center for Education Statistics (NCES). (2014a). Lessons learned from the 2012 Grade 4 writing computer-based assessment Study. Washington, DC: Institute of Education Sciences, U.S. Department of Education. Retrieved from: http://nces.ed.gov/nationsreportcard/writing/lessons/default.aspx. Accessed 19 Feb 2019.
- National Center for Education Statistics (NCES). (2014b). NAEP 2012 pilot writing computer-based assessment study–Grade 4: Technical memorandum. Washington, DC: Institute of Education Sciences, U.S. Department of Education.Google Scholar
- Peterson, S. S., & McClay, J. K. (2012). Assumptions and practices in using digital technologies to teach writing in middle-level classrooms across Canada. Literacy, 46(3), 140–146.Google Scholar
- Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift's electric factor analysis machine. Understanding Statistics: Statistical Issues in Psychology, Education, and the Social Sciences, 2(1), 13–43.Google Scholar
- Prior, P. (2006). A sociocultural theory of writing. In Handbook of writing research (pp. 54–66).Google Scholar
- Purcell, K., Buchanan, J., & Friedrich, L. (2013). The impact of digital tools on student writing and how writing is taught in schools. Washington, DC: Pew Research Center.Google Scholar
- Rogers, A., Stoeckel, J., & Sikali, E. (2013). National Assessment of Educational Progress (NAEP) 2011 writing assessment restricted-use data files data companion. Washington, DC: National Center for Education Statistics.Google Scholar
- Scribner, S., & Cole, M. (1981). The psychology of literacy (Vol. 198, No. 1). Cambridge, MA: Harvard University Press.Google Scholar
- Tate, T., & Warschauer, M. (in press). What’s in a Keystroke? Analysis of the 2011 NAEP Computer-Based 8th Grade Writing Assessment.Google Scholar
- Van Ittersum, D. (2011). Augmenting LITERACY: The role of expertise in digital writing. Composition Studies, 39(2), 61–77.Google Scholar
- Warschauer, M. (2006). Laptops and literacy: Learning in the wireless classroom. New York: Teachers College Press.Google Scholar
- Warschauer, M. (2011). Learning in the cloud: How (and why) to transform schools with digital media. New York: Teachers College Press.Google Scholar
- Wertsch, J. V. (1991). A sociocultural approach to socially shared cognition. Oxford: Oxford University Press.Google Scholar
- Wertsch, J. V. (1998). Mind as action. Oxford: Oxford University Press.Google Scholar
- Yim, S., Warschauer, M., Zheng, B., & Lawrence, J. (2014). Cloud-Based collaborative writing and the common core standards. Journal of Adolescent & Adult Literacy, 58(3), 243–254.Google Scholar
- Zheng, B., & Warschauer, M. (2015). Participation, interaction, and academic achievement in an online discussion environment. Computers & Education, 84, 78–89.Google Scholar
- Zwick, R. (1987). Assessing the dimensionality of NAEP reading data. Journal of Educational Measurement, 24(4), 293–300.Google Scholar