Skip to main content

Classification of Writing Patterns Using Keystroke Logs

Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS,volume 167)

Abstract

Keystroke logs are a valuable tool for writing research. Using large samples of student responses to two prompts targeting different writing purposes, we analyzed the longest 25 inter-word intervals in each keystroke log. The logs were extracted using the ETS keystroke logging engine. We found two distinct patterns of student writing processes associated with stronger and weaker writers, and an overall moderate association between the inter-word interval information and the quality of final product. The results suggest promise for the use of keystroke log analysis as a tool for describing patterns or styles of student writing processes.

Keywords

  • Keystroke logs
  • Writing processes
  • Writing pattern
  • Inter-word interval

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-38759-8_23
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-38759-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   139.00
Price excludes VAT (USA)
Hardcover Book
USD   139.00
Price excludes VAT (USA)
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    On one hand, the bin size needs to be large enough so that there are enough keystrokes in each bin. On the other hand, it needs to be small enough to show variations across bins. After a number of experiments, we found that ten bins are optimal.

References

  • Almond, R., Deane, P., Quinlan, T., & Wagner, M. (2012). A preliminary analysis of keystroke log data from a timed writing task (RR-12-23). Princeton, NJ: ETS Research Report.

    Google Scholar 

  • Alves, R. A., Castro, S. L., & de Sousa, L. (2007). Influence of typing skill on pause–execution cycles in written composition. In G. Rijlaarsdam (Series Ed.), M. Torrance, L. van Waes, & D. Galbraith (Vol. Eds.), Writing and cognition: Research and applications (Studies in Writing, Vol. 20, pp. 55–65). Amsterdam: Elsevier.

    Google Scholar 

  • Baaijen, V. M., Galbraith, D., & de Glopper, K. (2012). Keystroke analysis: Reflections on procedures and measures. Written Communications, 29, 246–277.

    CrossRef  Google Scholar 

  • Banerjee, R., Feng, S., Kang, J. S., & Choi, Y. (2014). Keystroke patterns as prosody in digital writings: A case study with deceptive reviews and essays. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar.

    Google Scholar 

  • Beauvais, C., Olive, T., & Passerault, J. (2011). Why are some texts good and others not? Relationship between text quality and management of the writing processes. Journal of Educational Psychology, 103, 415–428.

    CrossRef  Google Scholar 

  • Bennett, R. E. (2010). Cognitively Based Assessment of, for, and as Learning (CBAL): A preliminary theory of action for summative and formative assessment. Measurement, 8, 70–91.

    Google Scholar 

  • Bennett, R. E., Deane, P., van Rijn, P. (2016). From cognitive-domain theory to assessment practice. Educational Psychologist, 51, 82–107.

    Google Scholar 

  • Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing: Generating text in L1 and L2. Written Communication, 18, 80–98.

    CrossRef  Google Scholar 

  • Chukharev-Hudilainen, E. (2014). Pauses in spontaneous written communication: A keystroke logging study. Journal of Writing Research, 6, 61–84.

    CrossRef  Google Scholar 

  • Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.

    CrossRef  Google Scholar 

  • Deane, P. (2014). Using writing process and product features to assess writing quality and explore how those features relate to other literacy tasks (RR-14-03). Princeton, NJ: ETS Research Report.

    Google Scholar 

  • Deane, P., Sabatini, J. S., Feng, G., Sparks, J., Song, Y., Fowles, M., et al. (2015). Key practices in the English Language Arts (ELA): Linking learning theory, assessment, and instruction (RR-15-17). Princeton, NJ: ETS Research Report.

    Google Scholar 

  • Deane, P., & Zhang, M. (2015). Exploring the feasibility of using writing process features to assess text production skills (RR-15-26). Princeton, NJ: ETS Research Report.

    Google Scholar 

  • Dragsted, B., & Carl, M. (2013). Towards a classification of translation styles based on eye-tracking and keylogging data. Journal of Writing Research, 5, 133–158.

    CrossRef  Google Scholar 

  • Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613–619.

    CrossRef  Google Scholar 

  • Gould, J. D. (1980). Experiments on composing letters: Some facts, some myths, and some observations. In L. Gregg & E. Steinberg (Eds.), Cognitive processes in writing (pp. 97–127). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Grabowski, J. (2008). The internal structure of university students’ keyboard skills. Journal of Writing Research, 1, 27–52.

    CrossRef  Google Scholar 

  • Hao, J., Smith, L., Mislevy, R., von Davier, A., & Bauer, M. (2016). Taming log files from game and simulation-based assessment: Data model and data analysis tool. (RR-16-10) Princeton, NJ: ETS Research Report.

    Google Scholar 

  • Johnson, S. C. (1967). Hierarchical clustering schemes. Psychometrika, 32, 241–254.

    CrossRef  Google Scholar 

  • Jones, E., Oliphant, T., & Peterson, P. (2014). SciPy: Open source scientific tools for Python [Computer software]. Retrieved from http://www.scipy.org/.

  • Kalbfleisch, J. D., & Prentice, R. L. (2002). The statistical analysis of failure time data (2nd ed.). Hoboken, NJ: Wiley.

    CrossRef  MATH  Google Scholar 

  • Leijten, M., Macken, L., Hoste, V., van Horenbeeck, E., & van Waes, L. (2012). From character to word level: Enabling the linguistic analyses of Inputlog process data. Proceedings of the EACL 2012 Workshop on Computational Linguistics and Writing, Avignon, France.

    Google Scholar 

  • Leijten, M., & van Waes, L. (2013). Keystroke logging in writing research using Inputlog to analyze and visualize writing processes. Written Communication, 30, 358–392.

    CrossRef  Google Scholar 

  • Leijten, M., van Waes, L., Schriver, K., & Hayes, J. R. (2014). Writing in the workplace: Constructing documents using multiple digital sources. Journal of Writing Research, 5, 285–377.

    CrossRef  Google Scholar 

  • Miller, K. S. (2000). Academic writers on-line: Investigating pausing in the production of text. Language Teaching Research, 4, 123–148.

    CrossRef  Google Scholar 

  • Roca de Larios, J., Manchon, R., Murphy, L., & Marin, J. (2008). The foreign language writer’s strategic behavior in the allocation of time to writing processes. Journal of Second Language Writing, 17, 30–47.

    CrossRef  Google Scholar 

  • Ulrich, R., & Miller, J. (1993). Information processing models generating log normally distributed reaction times. Journal of Mathematical Psychology, 37, 513–525.

    CrossRef  MATH  Google Scholar 

  • van der Linden, W. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204.

    CrossRef  Google Scholar 

  • van Waes, L., Leijten, M., & van Weijen, D. (2009). Keystroke logging in writing research: Observing writing processes with Inputlog. GFI-Journal, No 2-3.

    Google Scholar 

  • Xu, X., & Ding, Y. (2014). An exploratory study of pauses in computer-assisted EFL writing. Language Learning & Technology, 18, 80–96.

    Google Scholar 

  • Zhang, M., & Deane, P. (2015). Process features in writing: Internal structure and incremental value over product features (RR-15-27). Princeton, NJ: ETS Research Report.

    Google Scholar 

Download references

Acknowledgements

We would like to thank Marie Wiberg, Don Powers, Gary Feng, Tanner Jackson, and Andre Rupp for their technical and editorial suggestions for this manuscript, thank Randy Bennett for his support of the study, and thank Shelby Haberman for his advice on the statistical analyses in this study.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mo Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Zhang, M., Hao, J., Li, C., Deane, P. (2016). Classification of Writing Patterns Using Keystroke Logs. In: van der Ark, L., Bolt, D., Wang, WC., Douglas, J., Wiberg, M. (eds) Quantitative Psychology Research. Springer Proceedings in Mathematics & Statistics, vol 167. Springer, Cham. https://doi.org/10.1007/978-3-319-38759-8_23

Download citation