An Approach to Measure Coding Competency Evolution

Toward Learning Analytics
  • Vive KumarEmail author
  • Kinshuk
  • Thamaraiselvi Somasundaram
  • Steve Harris
  • David Boulanger
  • Jeremie Seanosky
  • Geetha Paulmani
  • Karthikeyan Panneerselvam
Part of the Lecture Notes in Educational Technology book series (LNET)


There is a great deal of interest in the area of learning analytics and their use in assessing both student and content success in a digital learning environment. This chapter describes the results of a pilot study that used a combination of software tools and processes to collect the data generated from students taking an introductory C programming course, and their interactions related to specific study activities. This study helps determine whether it is possible to collect enough useful analytics to begin to identify constructive learning practices and strategies that determine competency evolution. We look at the types of learning traces collected through our system and then discuss how the data might be used to provide insight into student, class, or content actions. We then consider how additional analytics may be collected and used to supplement our initial results. The technologies used in this study address requirements of big data learning analytics where the data come from (a) the learner and his/her immediate surroundings; (b) the social network of the learner related to the learning tasks; and (c) the environment that inherently encases learning activities. While the data are used to measure competencies and their evolution over a period of time, the resultant profiles show the evolutionary processes students have adopted in developing specific competencies.


Learning analytics Big data Coding Novice programmer 



We gratefully acknowledge funding support from NSERC Canada, Athabasca University, and Shastri Indo-Canadian Institute for this research.


  1. Abdous, M., He, W., & Yen, C. J. (2013). Using data mining for predicting relationships between online question theme and final grade. Educational Technology and Society, 15(3), 77–88.Google Scholar
  2. Allen, J. F., Guinn, C. I., & Horvitz, E. (1999). Mixed-initiative interaction. IEEE Intelligent Systems, 14(5), 14–23.Google Scholar
  3. Angelo, T. A., Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers, (2nd ed.). San-Francisco: Jossey-Bass.Google Scholar
  4. Baghaei, N., Mitrovic, A., & Irwin, W. (2007). Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams. International Journal of CSCL, 2(2–3), 150–190.Google Scholar
  5. Bienkowski, M., Brecht, J., & Klo, J. (2012). The learning registry: Building a foundation for learning resource analytics. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 208–211).Google Scholar
  6. Boekaerts, M., Corno L. (2005). Self-regulation in the classroom: A perspective on assessment and interventions. Applied Psychology: An International Review, 54, 199–231.Google Scholar
  7. Brooks, C., Winter, M., Greer, J., & McCalla, G. (2004). The massive user modelling system. In The Proceedings of the 7th International Conference on Intelligent Tutoring Systems, Maceio, Brazil (pp. 635–645).Google Scholar
  8. Butler, D. L., Schnellert, L., & Higginson, S. (2008). Fostering agency and co-regulation: Teachers using formative assessment to calibrate practice in an age of accountability. Paper presented at the American Educational Research Association, April 2008.Google Scholar
  9. Cross, K. P., Angelo, T. A. (1988). Classroom assessment techniques: A handbook for faculty. Ann Arbor: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.Google Scholar
  10. Dicheva, D., Mizoguchi R., & Greer J. (2009). Semantic web technologies for e-learning. IOS Press, ISBN 978-1-60750-062-9.Google Scholar
  11. Essa, A., & Ayad, H. (2012). Improving student success using predictive models and data visualisations. In Proceedings of the ALT-C 2012 Conference (pp. 58–70).Google Scholar
  12. Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning (IJTEL), 4(5/6), 304–317.CrossRefGoogle Scholar
  13. Ghidini, C., Pammer, V., Scheir, P., Serafini, L., & Lindstaedt, S. (2007). APOSDLE: Learn@work with semantic web technology. In Proceedings of the International Conference on Knowledge Management (I-Know ’07), Graz, Austria.Google Scholar
  14. Hung, J. L., Hsu, Y. C., & Rice, K. (2012). Integrating data mining in program evaluation of K-12 online education. Educational Technology and Society, 15(3), 27–41.Google Scholar
  15. Jovanović, J., Gašević, D., Torniai, C., Bateman, S., & Hatala, M. (2009). The social semantic web in intelligent learning environments-state of the art and future challenges. Interactive Learning Environments, 17(4), 273–308.CrossRefGoogle Scholar
  16. Kim, M., & Lee, E. (2013). A multidimensional analysis tool for visualizing online interactions. Educational Technology and Society, 15(3), 89–102.Google Scholar
  17. Kosba, E., Dimitrova, V., Boyle, R. (2005). Using student and group models to support teachers in web-based distance education. In Proceedings of the 10th International Conference on User Modeling, Edinburgh, UK (pp. 124–133).Google Scholar
  18. Manyika, J., Chui M., Brown, B., Bughin, J., Dobbs, R., & Roxburgh, C. (2011). Big data: the next frontier for innovation, competition, and productivity. Report by McKinsey Global Institute. Retrieved January 10, 2014, from
  19. Mazza, R., Dimitrova, V. (2003). CourseVis: Externalising student information to facilitate instructors in distance learning. In Proceedings of the International Conference in Artificial Intelligence in Education (AIED 2003) (pp. 279–286).Google Scholar
  20. Mazza, R., Dimitrova, V. (2004). Visualising student tracking data to support instructors in web-based distance education. In Proceedings of the 13th World Wide Web Conference, NY, USA (pp. 154–161).Google Scholar
  21. Mazza, R., Milani, C. (2005). Exploring usage analysis in learning systems: gaining insights from visualisations. In Proceedings of the Workshop on Usage analysis in learning systems at the 12th International Conference on Artificial Intelligence in Education, Amsterdam, The Netherlands (pp. 65–72).Google Scholar
  22. Medeiros, F., Gomes, A. S., Amorim, R., & Medeiros, G. (2013). Architecture for social interactions monitoring in collaborative learning environments as a support for the teacher’s awareness. In Proceedings of International Conference on Advanced Learning Technologies (ICALT) (pp. 123–127).Google Scholar
  23. Rao, S., Kumar, V., Hatala, M., Gašević, D. (2007). Mixed-initiative interfaces to recognize regulate and reflect programming styles. In Proceedings. of the Workshop on AI for Human Computing at 20th International Joint Conference on Artificial Intelligence (IJCAI), India (pp. 71–78).Google Scholar
  24. Romero, C., Ventura, S., & Garcia, E. (2008). Data mining in course management systems: Moodle cases study and tutorial. Computers and Education, 51(1), 368–384.CrossRefGoogle Scholar
  25. Rountree, N., Rountree, J., Robins, A., Hannah, R. (2004). Interacting factors that predict success and failure in a CS1 course. ACM SIGCSE Bulletin, 36(4), 101–104.Google Scholar
  26. Shakya, J. (2005). Knowledge engineering and knowledge dissemination in a mixed-initiative ontological framework. MSc thesis, Simon Fraser University, Canada.Google Scholar
  27. Shum, S. B., & Ferguson, R. (2012). Social learning analytics. Educational Technology and Society, 15(3), 3–26.Google Scholar
  28. Watson, C., Li, F. W. B., & Godwin, J. L. (2013). Predicting performance in an introductory programming course by logging and analyzing student programming behavior. In Proceedings of International Conference on Advanced Learning Technologies (ICALT) (pp. 319–323).Google Scholar
  29. Wood, E., Zivcakova, L., Gentile, P., Archer, K., Pasquale, D. D., & Nosko, A. (2013). Examining the impact of off-task multi-tasking with technology on real-time classroom learning. Computers and Education, 58(2011), 365–374.Google Scholar
  30. Zinn, C., and Scheuer, O. (2006). Getting to know your student in distance-learning contexts. In Proceedings of the 1st European Conference on Technology Enhanced Learning (pp. 437–451).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Vive Kumar
    • 1
    Email author
  • Kinshuk
    • 1
  • Thamaraiselvi Somasundaram
    • 2
  • Steve Harris
    • 1
  • David Boulanger
    • 1
  • Jeremie Seanosky
    • 1
  • Geetha Paulmani
    • 1
  • Karthikeyan Panneerselvam
    • 2
  1. 1.School of Computing and Information SystemsAthabasca UniversityAthabascaCanada
  2. 2.Madras Institute of TechnologyAnna UniversityChennaiIndia

Personalised recommendations