Advertisement

Educational Technology Research and Development

, Volume 64, Issue 6, pp 1083–1106 | Cite as

Uncovering student learning profiles with a video annotation tool: reflective learning with and without instructional norms

  • Negin MirriahiEmail author
  • Daniyal Liaqat
  • Shane Dawson
  • Dragan Gašević
Research Article

Abstract

This study explores the types of learning profiles that evolve from student use of video annotation software for reflective learning. The data traces from student use of the software were analysed across four undergraduate courses with differing instructional conditions. That is, the use of graded or non-graded self-reflective annotations. Using hierarchical cluster analysis, four profiles of students emerged: minimalists, task-oriented, disenchanted, and intensive users. Students enrolled in one of the courses where grading of the video annotation software was present, were exposed to either another graded course (annotations graded) or non-graded course (annotations not graded) in their following semester of study. Further analysis revealed that in the presence of external factors (i.e., grading), more students fell within the task-oriented and intensive clusters. However, when the external factor is removed, most students exhibited the disenchanted and minimalist learning behaviors. The findings provide insight into how students engage with the different features of a video annotation tool when there are graded or non-graded annotations and, most importantly, that having experience with one course where there are external factors influencing students’ use of the tool is not sufficient to sustain their learning behaviour in subsequent courses where the external factor is removed.

Keywords

Instructional norms Learning technology Video annotation Learning analytics Higher education 

Notes

Acknowledgments

This research is in part supported by Australian Office of Learning and Teaching (Innovation and Development Grant), Canada Research Chair Program of the Government of Canada, Social Sciences and Humanities Research Council of Canada (Insight Grant), and Natural Sciences and Engineering Research Council of Canada (Discovery Grant). We also thank Thomas Dang for data extraction.

References

  1. Alexander, P. A., Jetton, T. L., & Kulikowich, J. M. (1995). Interrelationship of knowledge, interest, and recall: Assessing a model of domain learning. Journal of Educational Psychology, 87(4), 559.CrossRefGoogle Scholar
  2. Al-Qahtani, A. A. Y., & Higgins, S. E. (2013). Effects of traditional, blended and e-learning on students’ achievement in higher education: E-Learning, blended and traditional learning. Journal of Computer Assisted Learning, 29(3), 220–234. doi: 10.1111/j.1365-2729.2012.00490.x.CrossRefGoogle Scholar
  3. Aubert, O., Prié, Y., & Canellas, C. (2014). Leveraging video annotations in video-based e-learning. In 7th International Conference on Computer Supported Education, Barcelona, Spain. Retrieved from http://arxiv.org/abs/1404.4607.
  4. Azevedo, R., Moos, D. C., Greene, J. A., Winters, F. I., & Cromley, J. G. (2008). Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia? Educational Technology Research and Development, 56(1), 45–72. doi: 10.1007/s11423-007-9067-0.CrossRefGoogle Scholar
  5. Barab, S. A., Bowdish, B. E., & Lawless, K. A. (1997). Hypermedia navigation: Profiles of hypermedia users. Educational Technology Research and Development, 45(3), 23–41.CrossRefGoogle Scholar
  6. Bargeron, D., Gupta, A., Grudin, J., & Sanocki, E. (1999). Annotations for streaming video on the Web: System design and usage studies. Computer Networks, 31(11), 1139–1153.CrossRefGoogle Scholar
  7. Beretvas, S. N., Meyers, J. L., & Leite, W. L. (2002). A reliability generalization study of the Marlowe-Crowne social desirability scale. Educational and Psychological Measurement, 62(4), 570–589. doi: 10.1177/0013164402062004003.CrossRefGoogle Scholar
  8. Brooks, C., Epp, C., Logan, G., & Greer, J. (2011). The who, what, when, and why of lecture capture. In Proceedings of the First International Conference on Learning Analytics and Knowledge (pp. 86–92). Banff, Alberta, Canada: ACM. doi: 10.1145/2090116.2090128.
  9. Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014). Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis. Psychological Bulletin, 140(4), 980.CrossRefGoogle Scholar
  10. Chen, P.-S. D., Lambert, A. D., & Guidry, K. R. (2010). Engaging online learners: The impact of Web-based learning technology on college student engagement. Computers & Education, 54(4), 1222–1232. doi: 10.1016/j.compedu.2009.11.008.CrossRefGoogle Scholar
  11. Cleave, J. B., Edelson, D., & Beckwith, R. (1993). A matter of style: An analysis of student interaction with a computer-based learning environment. In American Educational Research Association (AERA) Annual Meeting, Atlanta, GA.Google Scholar
  12. Colasante, M. (2011). Using video annotation to reflect on and evaluate physical education pre-service teaching practice. Australasian Journal of Educational Technology, 27(1), 66–88.CrossRefGoogle Scholar
  13. Colasante, M., & Fenn, J. (2009). “mat”: A new media annotation tool with an interactive learning cycle for application in tertiary education. In World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2009, pp. 3546–3551). Retrieved from http://www.editlib.org/pv/31992.
  14. Cross, A., Bayyapunedi, M., Ravindran, D., Cutrell, E., & Thies, W. (2014). VidWiki: Enabling the crowd to improve the legibility of online educational videos (pp. 1167–1175). New York: ACM Press. doi: 10.1145/2531602.2531670.Google Scholar
  15. Dawson, S., Macfadyen, L., Evan, F. R., Foulsham, T., & Kingstone, A. (2012). Using technology to encourage self-directed learning: The Collaborative Lecture Annotation System (CLAS). In Ascilite conference (Vol. 2012). http://www.ascilite2012.org/images/custom/dawson,_shane_-_using_technology_to_encourage.pdf.
  16. del Valle, R., & Duffy, T. M. (2009). Online learning: Learner characteristics and their approaches to managing learning. Instructional Science, 37(2), 129–149. doi: 10.1007/s11251-007-9039-0.CrossRefGoogle Scholar
  17. Dunning, T. (2012). Natural experiments in the social sciences: A design-based approach. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  18. Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550.Google Scholar
  19. Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95–105. doi: 10.1016/j.iheduc.2004.02.001.CrossRefGoogle Scholar
  20. Garrison, D. R., & Vaughan, N. (2008). Blended learning in higher education. San Francisco: Jossey-Bass Inc., Publishers.Google Scholar
  21. Gašević, D., Mirriahi, N., & Dawson, S. (2014). Analytics of the effects of video use and instruction to support reflective learning. In Proceedings of the fourth international conference on learning analytics and Knowledge (pp 123–132). ACM Press. doi: 10.1145/2567574.2567590.
  22. Giannakos, M. N., Chorianopoulos, K., & Chrisochoides, N. (2014). Collecting and making sense of video learning analytics. In Frontiers in Education Conference (FIE), 2014 IEEE (pp. 1–7). IEEE. Retrieved from http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7044485.
  23. Giannakos, M. N., Chorianopoulos, K., & Chrisochoides, N. (2015). Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. The International Review of Research in Open and Distributed Learning, 16(1). Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1976.
  24. Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education, 1(1), 3–31.Google Scholar
  25. Gonyea, R. M. (2005). Self-reported data in institutional research: Review and recommendations. New Directions for Institutional Research, 2005(127), 73–89.CrossRefGoogle Scholar
  26. Gosper, M., Malfroy, J., & McKenzie, J. (2013). Students’ experiences and expectations of technologies: An Australian study designed to inform planning and development decisions. Australasian Journal of Educational Technology, 29(2). Retrieved from http://ascilite.org.au/ajet/submission/index.php/AJET/article/view/127.
  27. Graham, C. R., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. The Internet and Higher Education, 18, 4–14. doi: 10.1016/j.iheduc.2012.09.003.CrossRefGoogle Scholar
  28. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57.Google Scholar
  29. Hadwin, A. F., Nesbit, J. C., Jamieson-Noel, D., Code, J., & Winne, P. H. (2007). Examining trace data to explore self-regulated learning. Metacognition and Learning, 2(2–3), 107–124. doi: 10.1007/s11409-007-9016-7.CrossRefGoogle Scholar
  30. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112. doi: 10.3102/003465430298487.CrossRefGoogle Scholar
  31. Hulsman, R. L., Harmsen, A. B., & Fabriek, M. (2009). Reflective teaching of medical communication skills with DiViDU: Assessing the level of student reflection on recorded consultations with simulated patients. Patient Education and Counseling, 74(2), 142–149. doi: 10.1016/j.pec.2008.10.009.CrossRefGoogle Scholar
  32. Koestner, R., Ryan, R. M., Bernieri, F., & Holt, K. (1984). Setting limits on children’s behavior: The differential effects of controlling vs. informational styles on intrinsic motivation and creativity. Journal of Personality, 52(3), 233–248. doi: 10.1111/j.1467-6494.1984.tb00879.x.CrossRefGoogle Scholar
  33. Lust, G., Elen, J., & Clarebout, G. (2013). Regulation of tool-use within a blended course: Student differences and performance effects. Computers & Education, 60(1), 385–395. doi: 10.1016/j.compedu.2012.09.001.CrossRefGoogle Scholar
  34. Lust, G., Vandewaetere, M., Ceulemans, E., Elen, J., & Clarebout, G. (2011). Tool-use in a blended undergraduate course. In search of user profiles. Computers & Education, 57(3), 2135–2144. doi: 10.1016/j.compedu.2011.05.010.CrossRefGoogle Scholar
  35. Magenheim, J., Reinhardt, W., Roth, A., Moi, M., & Engbring, D. (2010). Integration of a video annotation tool into a coactive learning and working environment. In Key Competencies in the Knowledge Society (pp. 257–268). Springer. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642-15378-5_25.
  36. McGill, T. J., & Klobas, J. E. (2009). A task–technology fit view of learning management system impact. Computers & Education, 52(2), 496–508. doi: 10.1016/j.compedu.2008.10.002.CrossRefGoogle Scholar
  37. Mu, X. (2010). Towards effective video annotation: An approach to automatically link notes with video content. Computers & Education, 55(4), 1752–1763. doi: 10.1016/j.compedu.2010.07.021.CrossRefGoogle Scholar
  38. Perkins, D. N. (1985). The fingertip effect: How information-processing technology shapes thinking. Educational Researcher, 14(7), 11–17. doi: 10.3102/0013189X014007011.CrossRefGoogle Scholar
  39. Phillips, R., Maor, D., Preston, G., & Cumming-Potvin, W. (2012). Exploring learning analytics as indicators of study behaviour. In Proceedings of EdMedia: World Conference on Educational Media and Technology (pp. 2861–2867). Denver, Colorado, USA: Association for the Advancement of Computing in Education. Retrieved from http://researchrepository.murdoch.edu.au/10460/.
  40. Pintrich, P. R., & de Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40. doi: 10.1037/0022-0663.82.1.33.CrossRefGoogle Scholar
  41. Rich, P. J., & Hannafin, M. (2008). Video annotation tools: Technologies to scaffold, structure, and transform teacher reflection. Journal of Teacher Education, 60(1), 52–67. doi: 10.1177/0022487108328486.CrossRefGoogle Scholar
  42. Risko, E. F., Foulsham, T., Dawson, S., & Kingstone, A. (2013). The Collaborative Lecture Annotation System (CLAS): A new tool for distributed learning. IEEE Transactions on Learning Technologies, 6(1), 4–13. doi: 10.1109/TLT.2012.15.CrossRefGoogle Scholar
  43. Ryan, R. M., Mims, V., & Koestner, R. (1983). Relation of reward contingency and interpersonal context to intrinsic motivation: A review and test using cognitive evaluation theory. Journal of Personality and Social Psychology, 45(4), 736–750. doi: 10.1037/0022-3514.45.4.736.CrossRefGoogle Scholar
  44. Torrisi-Steele, G., & Drew, S. (2013). The literature landscape of blended learning in higher education: the need for better understanding of academic blended practice. International Journal for Academic Development,. doi: 10.1080/1360144X.2013.786720.Google Scholar
  45. Ward, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58(301), 236–244. doi: 10.1080/01621459.1963.10500845.CrossRefGoogle Scholar
  46. Winne, P. H. (2006). How software technologies can improve research on learning and bolster school reform. Educational Psychologist, 41(1), 5–17. doi: 10.1207/s15326985ep4101_3.CrossRefGoogle Scholar
  47. Winne, P. H. (2013). Learning strategies, study skills, and self-regulated learning in postsecondary education. In M. B. Paulsen (Ed.) Higher education: Handbook of theory and research (Vol. 28, pp. 377–403). Dordrecht: Springer. Retrieved from http://link.springer.com/chapter/10.1007%2F978-94-007-5836-0_8#.
  48. Winne, P. H., & Jamieson-Noel, D. (2002). Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology, 27(4), 551–572.CrossRefGoogle Scholar
  49. Yen, J.-C., & Lee, C.-Y. (2011). Exploring problem solving patterns and their impact on learning achievement in a blended learning environment. Computers & Education, 56(1), 138–145. doi: 10.1016/j.compedu.2010.08.012.CrossRefGoogle Scholar
  50. Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Thousand Oaks, CA: Sage Publications Inc.Google Scholar
  51. Yousef, A. M. F., Chatti, M. A., & Schroeder, U. (2014). Video-based learning: A critical analysis of the research published in 2003–2013 and future visions. In eLmL 2014, The Sixth International Conference on Mobile, Hybrid, and On-line Learning (pp. 112–119). Retrieved from http://www.thinkmind.org/index.php?view=article&articleid=elml_2014_5_30_50050.

Copyright information

© Association for Educational Communications and Technology 2016

Authors and Affiliations

  • Negin Mirriahi
    • 1
    Email author
  • Daniyal Liaqat
    • 2
  • Shane Dawson
    • 3
  • Dragan Gašević
    • 4
    • 5
  1. 1.School of Education & Learning and Teaching UnitUNSWSydneyAustralia
  2. 2.Department of Computer ScienceUniversity of TorontoTorontoCanada
  3. 3.Teaching Innovation UnitUniversity of South AustraliaAdelaideAustralia
  4. 4.Moray House School of EducationUniversity of EdinburghEdinburghUK
  5. 5.School of InformaticsUniversity of EdinburghEdinburgh, MidlothianUK

Personalised recommendations