Abstract
Keystroke logs are a valuable tool for writing research. Using large samples of student responses to two prompts targeting different writing purposes, we analyzed the longest 25 inter-word intervals in each keystroke log. The logs were extracted using the ETS keystroke logging engine. We found two distinct patterns of student writing processes associated with stronger and weaker writers, and an overall moderate association between the inter-word interval information and the quality of final product. The results suggest promise for the use of keystroke log analysis as a tool for describing patterns or styles of student writing processes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
On one hand, the bin size needs to be large enough so that there are enough keystrokes in each bin. On the other hand, it needs to be small enough to show variations across bins. After a number of experiments, we found that ten bins are optimal.
References
Almond, R., Deane, P., Quinlan, T., & Wagner, M. (2012). A preliminary analysis of keystroke log data from a timed writing task (RR-12-23). Princeton, NJ: ETS Research Report.
Alves, R. A., Castro, S. L., & de Sousa, L. (2007). Influence of typing skill on pause–execution cycles in written composition. In G. Rijlaarsdam (Series Ed.), M. Torrance, L. van Waes, & D. Galbraith (Vol. Eds.), Writing and cognition: Research and applications (Studies in Writing, Vol. 20, pp. 55–65). Amsterdam: Elsevier.
Baaijen, V. M., Galbraith, D., & de Glopper, K. (2012). Keystroke analysis: Reflections on procedures and measures. Written Communications, 29, 246–277.
Banerjee, R., Feng, S., Kang, J. S., & Choi, Y. (2014). Keystroke patterns as prosody in digital writings: A case study with deceptive reviews and essays. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.
Beauvais, C., Olive, T., & Passerault, J. (2011). Why are some texts good and others not? Relationship between text quality and management of the writing processes. Journal of Educational Psychology, 103, 415–428.
Bennett, R. E. (2010). Cognitively Based Assessment of, for, and as Learning (CBAL): A preliminary theory of action for summative and formative assessment. Measurement, 8, 70–91.
Bennett, R. E., Deane, P., van Rijn, P. (2016). From cognitive-domain theory to assessment practice. Educational Psychologist, 51, 82–107.
Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing: Generating text in L1 and L2. Written Communication, 18, 80–98.
Chukharev-Hudilainen, E. (2014). Pauses in spontaneous written communication: A keystroke logging study. Journal of Writing Research, 6, 61–84.
Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.
Deane, P. (2014). Using writing process and product features to assess writing quality and explore how those features relate to other literacy tasks (RR-14-03). Princeton, NJ: ETS Research Report.
Deane, P., Sabatini, J. S., Feng, G., Sparks, J., Song, Y., Fowles, M., et al. (2015). Key practices in the English Language Arts (ELA): Linking learning theory, assessment, and instruction (RR-15-17). Princeton, NJ: ETS Research Report.
Deane, P., & Zhang, M. (2015). Exploring the feasibility of using writing process features to assess text production skills (RR-15-26). Princeton, NJ: ETS Research Report.
Dragsted, B., & Carl, M. (2013). Towards a classification of translation styles based on eye-tracking and keylogging data. Journal of Writing Research, 5, 133–158.
Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613–619.
Gould, J. D. (1980). Experiments on composing letters: Some facts, some myths, and some observations. In L. Gregg & E. Steinberg (Eds.), Cognitive processes in writing (pp. 97–127). Hillsdale, NJ: Lawrence Erlbaum.
Grabowski, J. (2008). The internal structure of university students’ keyboard skills. Journal of Writing Research, 1, 27–52.
Hao, J., Smith, L., Mislevy, R., von Davier, A., & Bauer, M. (2016). Taming log files from game and simulation-based assessment: Data model and data analysis tool. (RR-16-10) Princeton, NJ: ETS Research Report.
Johnson, S. C. (1967). Hierarchical clustering schemes. Psychometrika, 32, 241–254.
Jones, E., Oliphant, T., & Peterson, P. (2014). SciPy: Open source scientific tools for Python [Computer software]. Retrieved from http://www.scipy.org/.
Kalbfleisch, J. D., & Prentice, R. L. (2002). The statistical analysis of failure time data (2nd ed.). Hoboken, NJ: Wiley.
Leijten, M., Macken, L., Hoste, V., van Horenbeeck, E., & van Waes, L. (2012). From character to word level: Enabling the linguistic analyses of Inputlog process data. Proceedings of the EACL 2012 Workshop on Computational Linguistics and Writing, Avignon, France.
Leijten, M., & van Waes, L. (2013). Keystroke logging in writing research using Inputlog to analyze and visualize writing processes. Written Communication, 30, 358–392.
Leijten, M., van Waes, L., Schriver, K., & Hayes, J. R. (2014). Writing in the workplace: Constructing documents using multiple digital sources. Journal of Writing Research, 5, 285–377.
Miller, K. S. (2000). Academic writers on-line: Investigating pausing in the production of text. Language Teaching Research, 4, 123–148.
Roca de Larios, J., Manchon, R., Murphy, L., & Marin, J. (2008). The foreign language writer’s strategic behavior in the allocation of time to writing processes. Journal of Second Language Writing, 17, 30–47.
Ulrich, R., & Miller, J. (1993). Information processing models generating log normally distributed reaction times. Journal of Mathematical Psychology, 37, 513–525.
van der Linden, W. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204.
van Waes, L., Leijten, M., & van Weijen, D. (2009). Keystroke logging in writing research: Observing writing processes with Inputlog. GFI-Journal, No 2-3.
Xu, X., & Ding, Y. (2014). An exploratory study of pauses in computer-assisted EFL writing. Language Learning & Technology, 18, 80–96.
Zhang, M., & Deane, P. (2015). Process features in writing: Internal structure and incremental value over product features (RR-15-27). Princeton, NJ: ETS Research Report.
Acknowledgements
We would like to thank Marie Wiberg, Don Powers, Gary Feng, Tanner Jackson, and Andre Rupp for their technical and editorial suggestions for this manuscript, thank Randy Bennett for his support of the study, and thank Shelby Haberman for his advice on the statistical analyses in this study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhang, M., Hao, J., Li, C., Deane, P. (2016). Classification of Writing Patterns Using Keystroke Logs. In: van der Ark, L., Bolt, D., Wang, WC., Douglas, J., Wiberg, M. (eds) Quantitative Psychology Research. Springer Proceedings in Mathematics & Statistics, vol 167. Springer, Cham. https://doi.org/10.1007/978-3-319-38759-8_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-38759-8_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-38757-4
Online ISBN: 978-3-319-38759-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)