Learning Traces, Competence Assessment, and Causal Inference for English Composition

  • Clayton Clemens
  • Vivekanandan KumarEmail author
  • David Boulanger
  • Jérémie Seanosky
  • Kinshuk
Part of the Lecture Notes in Educational Technology book series (LNET)


It is widely acknowledged that writing is a process and should be taught as a process. However, it is still assessed as though it is a product. Educational technology makes now possible for teachers to become observers of the writing process of their students to discover how their writing competences (e.g., grammatical accuracy, topic flow, transition, and vocabulary usage) develop over time. The present research proposes an innovative technique to identify the actual drivers of writing performance through a formal causality framework, unleashing a new source of potential insights to scaffold more effectively the writing process and guarantee more reliable success at the end.


Analytics of writing process Causality Competence Big data Natural-language processing Learning analytics 



This research is supported by the Industrial Research Chair and Discovery programs of the Natural Sciences and Engineering Research Council of Canada, and the internal research funding programs of Athabasca University, Canada.


  1. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238.CrossRefGoogle Scholar
  2. Boulanger, D., Seanosky, J., Clemens, C., Kumar, V., & Kinshuk. (2016). SCALE: A smart competence analytics solution for English writing. In Proceedings of the 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT) (pp. 468–472).
  3. Boulanger, D., Seanosky, J., Pinnell, C., Bell, J., Kumar, V., & Kinshuk. (2016). SCALE: A competence analytics framework. In Y. Li, M. Chang, M. Kravcik, E. Popescu, R. Huang, Kinshuk, & N.-S. Chen (Eds.), State-of-the-art and future directions of smart learning (pp. 19–30). Singapore: Springer. Scholar
  4. Cohen, J., Dolan, B., Dunlap, M., Hellerstein, J. M., & Welton, C. (2009). MAD skills: New analysis practices for big data. Proceedings of the VLDB Endowment, 2(2), 1481–1492. Scholar
  5. Clemens, C. (2017). A causal model of writing competence (Master’s thesis). Retrieved from
  6. Cramir, H. (1946). Mathematical methods of statistics. Princeton: Princeton University Press.Google Scholar
  7. Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4), 507–521. Retrieved from Scholar
  8. Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221.CrossRefGoogle Scholar
  9. Geiger, D., Paz, A., & Pearl, J. (1991). Axioms and algorithms for inferences involving probabilistic independence. Information and Computation, 91(1), 128–141. Scholar
  10. Guillot, C., Guillot, R., Kumar, V., & Kinshuk. (2016). MUSIX: Learning analytics in music teaching. In Y. Li, M. Chang, M. Kravcik, E. Popescu, R. Huang, Kinshuk, & N.-S. Chen (Eds.), State-of-the-Art and Future Directions of Smart Learning (pp. 269–273). Singapore: Springer. Scholar
  11. Johnson, P. M., Kou, H., Agustin, J., Chan, C., Moore, C., Miglani, J., … Doane, W. E. J. (2003). Beyond the personal software process: Metrics collection and analysis for the differently disciplined. In Proceedings of the 25th International Conference on Software Engineering, 2003 (pp. 641–646).
  12. Kincaid, J. P., Fishburne, R. P., Jr., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel.Google Scholar
  13. Klein, D., & Manning, C. D. (2003). Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics (Vol. 1, pp. 423–430). Stroudsburg, PA, USA: Association for Computational Linguistics.
  14. Kumar, V., Kinshuk, Somasundaram, T., Harris, S., Boulanger, D., Seanosky, J., … Panneerselvam, K. (2015). An approach to measure coding competency evolution. In M. Chang & Y. Li (Eds.), Smart learning environments (pp. 27–43). Berlin, Heidelberg: Springer. Scholar
  15. Lewkow, N., Feild, J., Zimmerman, N., Riedesel, M., Essa, A., Boulanger, D., … Kode, S. (2016). A scalable learning analytics platform for automated writing feedback. In Proceedings of the 3rd (2016) ACM Conference on Learning @ Scale (pp. 109–112). New York, NY, USA: ACM.
  16. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130.CrossRefGoogle Scholar
  17. Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39–41. Scholar
  18. Murray, D. M. (1972). Teach writing as a process not product. The Leaflet, 71, 11–14.Google Scholar
  19. Nasukawa, T., & Yi, J. (2003). Sentiment analysis: Capturing favorability using natural language processing. In Proceedings of the 2nd International Conference on Knowledge Capture (pp. 70–77).Google Scholar
  20. Nesi, H., Sharpling, G., & Ganobcsik-Williams, L. (2004). Student papers across the curriculum: Designing and developing a corpus of British student writing. Computers and Composition, 21(4), 439–450. Scholar
  21. O’Rourke, S. T., Calvo, R. A., & McNamara, D. S. (2011). Visualizing topic flow in students’ essays. Educational Technology & Society, 14(3), 4–15.Google Scholar
  22. Pang, B., & Lee, L. (2008). Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135. Scholar
  23. Peirce, C. S., & Jastrow, J. (1884). On small differences in sensation.Google Scholar
  24. Russell, B. (1912). On the notion of cause. Proceedings of the Aristotelian Society, 13, 1–26. Retrieved from
  25. Sampson, D., & Fytros, D. (2008). Competence models in technology-enhanced competence-based learning. In H. H. Adelsberger, Kinshuk, J. M. Pawlowski, & D. G. Sampson (Eds.), Handbook on information technologies for education and training (pp. 155–177). Berlin, Heidelberg: Springer. Scholar
  26. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. Retrieved from Scholar
  27. Simpson, E. H. (1951). The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society. Series B (Methodological), 13(2), 238–241. Retrieved from
  28. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C., Ng, A., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).Google Scholar
  29. Spirtes, P., Glymour, C. N., & Scheines, R. (2000). Causation, prediction, and search. MIT press.Google Scholar
  30. Toutanova, K., Klein, D., Manning, C. D., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology (Vol. 1, pp. 173–180). Stroudsburg, PA, USA: Association for Computational Linguistics.
  31. Verhelst, N., Van Avermaet, P., Takala, S., Figueras, N., & North, B. (2009). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.Google Scholar
  32. Waes, L. V., & Schellens, P. J. (2003). Writing profiles: The effect of the writing mode on pausing and revision patterns of experienced writers. Journal of Pragmatics, 35(6), 829–853. Scholar
  33. Yi, J., Nasukawa, T., Bunescu, R., & Niblack, W. (2003). Sentiment analyzer: Extracting sentiments about a given topic using natural language processing techniques. In 3rd IEEE International Conference on Data Mining (pp. 427–434).

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Clayton Clemens
    • 1
  • Vivekanandan Kumar
    • 1
    Email author
  • David Boulanger
    • 1
  • Jérémie Seanosky
    • 1
  • Kinshuk
    • 2
  1. 1.Athabasca UniversityEdmontonCanada
  2. 2.University of North TexasDentonUSA

Personalised recommendations