Advertisement

A First Step Towards Synthesizing Rubrics and Video for the Formative Assessment of Complex Skills

  • Kevin AckermansEmail author
  • Ellen Rusman
  • Saskia Brand-Gruwel
  • Marcus Specht
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 653)

Abstract

For learners, it can be difficult to imagine how to perform a complex skill from textual information found in a text-based analytic rubric. In this paper we identify three deficiencies of the text-based analytic rubric for the formative assessment of complex skills. We propose to address the text-based analytic rubric’s deficiencies by adding video modeling examples. With the resulting Video Enhanced Rubric we aim to improve the formative assessment of complex skills by fostering learner’s mental model development, feedback quality and complex skill mastery.

Keywords

Video Rubrics (Formative) assessment Complex skills Mental models 

Notes

Acknowledgement

We would like to gratefully acknowledge the contribution of the Viewbrics project, that is funded by the practice-oriented research programme of the Netherlands Initiative for Education Research (NRO), part of The Netherlands Organisation for Scientific Research (NWO).

References

  1. 1.
    Panadero, E., Romero, M.: To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assess. Educ. Principles Policy Pract. 21, 133–148 (2014). doi: 10.1080/0969594X.2013.877872 CrossRefGoogle Scholar
  2. 2.
    Rusman, E., Martínez-Monés, A., Boon, J., et al.: Computer Assisted Assessment – Research into E-Assessment: Proceedings of International Conference, CAA 2014, Zeist, The Netherlands, June 30–July 1 2014. In: Kalz, M., Ras, E. (eds.), pp. 1–14. Springer International Publishing, Cham (2014)Google Scholar
  3. 3.
    Thijs, A., Fisser, P., van der Hoeven, M.: 21E Eeuwse Vaardigheden in Het Curriculum Van Het Funderend Onderwijs. Slo 128 (2014)Google Scholar
  4. 4.
    Janssen-Noordman, A.M., Van Merriënboer, J.J.G.: Innovatief Onderwijs Ontwerpen. Wolters-Noordhoff, Groningen (2002)Google Scholar
  5. 5.
    Van Merriënboer, J.J.G., Kester, L.: The four-component instructional design model: multimedia principles in environments for complex learning. In: The Cambridge Handbook of Multimedia Learning (2005). doi: 10.1017/CBO9781139547369.007
  6. 6.
    Van Merriënboer, J.J.G., Kirschner, P.A.: Ten Steps to Complex Learning. Lawrence Erlbaum Associates Inc., New Jersey (2007)Google Scholar
  7. 7.
    Jonsson, A., Svingby, G.: The use of scoring rubrics: reliability, validity and educational consequences. Educ. Res. Rev. 2, 130–144 (2007). doi: 10.1016/j.edurev.2007.05.002 CrossRefGoogle Scholar
  8. 8.
    Westera, W.: Reframing contextual learning: anticipating the virtual extensions of context 14, 201–212 (2011)Google Scholar
  9. 9.
    Matthews, W.J., Buratto, L.G., Lamberts, K.: Exploring the memory advantage for moving scenes. Vis. Cogn. 18, 1393–1420 (2010). doi: 10.1080/13506285.2010.492706 CrossRefGoogle Scholar
  10. 10.
    Panadero, E., Jonsson, A.: The use of scoring rubrics for formative assessment purposes revisited: a review. Educ. Res. Rev. 9, 129–144 (2013). doi: 10.1016/j.edurev.2013.01.002 CrossRefGoogle Scholar
  11. 11.
    Mertler, C.: Designing scoring rubrics for your classroom. Pract. Assess. Res. Eval. 7, 1–10 (2001)Google Scholar
  12. 12.
    Brookhart, S.M., Chen, F.: The quality and effectiveness of descriptive rubrics. Educ. Rev. 1–26 (2014). doi: 10.1080/00131911.2014.929565
  13. 13.
    Reynolds-Keefer, L.: Rubric-referenced assessment in teacher preparation: an opportunity to learn by using. Pract. Assess. Res. Eval. 15, 1–9 (2010)Google Scholar
  14. 14.
    Andrade, H.G.: The effects of instructional rubrics on learning to write. Curr. Issues Educ. 4, 1–39 (2001)Google Scholar
  15. 15.
    Schamber, J.F., Mahoney, S.L.: Assesing and improving the quality of group critical thinking exhibited in the final projects of collaborative learning groups. J. Gen. Educ. 55, 103–137 (2006). doi: 10.1353/jge.2006.0025 CrossRefGoogle Scholar
  16. 16.
    Andrade, H., Du, Y.: Student perspectives on rubric-referenced assessment. Pract. Assess. Res. Eval. 10, 1–11 (2005). doi: 10.1080/02602930801955986 Google Scholar
  17. 17.
    Good, T.L.: Two decades of research on teacher expectations: findings and future directions. J. Teach. Educ. 38, 32–47 (1987). doi: 10.1177/002248718703800406 CrossRefGoogle Scholar
  18. 18.
    Panadero, E., Tapia, J.A., Huertas, J.A.: Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learn. Individ. Differ. 22, 806–813 (2012). doi: 10.1016/j.lindif.2012.04.007 CrossRefGoogle Scholar
  19. 19.
    Wolters, C.A.: Regulation of motivation: evaluating an underemphasized aspect of self-regulated learning. Educ. Psychol. 38, 189–205 (2003). doi: 10.1207/S15326985EP3804_1
  20. 20.
    Kuhl, J.: A functional-design approach to motivation and self-regulation: the dynamics of personality systems and interactions. In: Handbook of Self-regulation, pp. 111–169 (2000)Google Scholar
  21. 21.
    Panadero, E., Romero, M., Strijbos, J.W.: The impact of a rubric and friendship on peer assessment: effects on construct validity, performance, and perceptions of fairness and comfort. Stud. Educ. Eval. 39, 195–203 (2013). doi: 10.1016/j.stueduc.2013.10.005 CrossRefGoogle Scholar
  22. 22.
    Gary, M.S., Wood, R.E.: Mental models, decision rules, and performance heterogeneity. Strateg. Manage. J. 32, 569–594 (2011). doi: 10.1002/smj.899 CrossRefGoogle Scholar
  23. 23.
    Kim, Y., McDonough, K.: Using pretask modelling to encourage collaborative learning opportunities. Lang. Teach. Res. 15, 183–199 (2011). doi: 10.1177/1362168810388711 CrossRefGoogle Scholar
  24. 24.
    De Grez, L., Valcke, M., Roozen, I.: The differential impact of observational learning and practice-based learning on the development of oral presentation skills in higher education. High. Educ. Res. Dev. 33, 256–271 (2014). doi: 10.1080/07294360.2013.832155 CrossRefGoogle Scholar
  25. 25.
    Frerejean, J., van Strien, J.L.H., Kirschner, P.A., Brand-Gruwel, S.: Completion strategy or emphasis manipulation? Task support for teaching information problem solving. Comput. Hum. Behav. 62, 90–104 (2015). doi: 10.1016/j.chb.2016.03.048. Manuscript SubmissionCrossRefGoogle Scholar
  26. 26.
    Andrade, H., Buff, C., Terry, J., et al.: Assessment-driven improvements in middle school students’ writing. Middle Sch. J. 40, 4–12 (2009)CrossRefGoogle Scholar
  27. 27.
    Brookhart, S.M., Chen, F.: The quality and effectiveness of descriptive rubrics. Educ. Rev. 1911, 1–26 (2014). doi: 10.1080/00131911.2014.929565 Google Scholar
  28. 28.
    Efklides, A.: Interactions of metacognition with motivation and affect in self-regulated learning: the MASRL model. Educ. Psychol. 46, 6–25 (2011). doi: 10.1080/00461520.2011.538645 CrossRefGoogle Scholar
  29. 29.
    Schunk, D.H., Usher, E.L.: Assessing self-efficacy for self-regulated learning. In: Handbook of Self-Regulation of Learning and Performance, pp. 282–297 (2011)Google Scholar
  30. 30.
    Zimmerman, B.J., Kitsantas, A.: Acquiring writing revision and self-regulatory skill through observation and emulation. J. Educ. Psychol. 94, 660–668 (2002). doi: 10.1037/0022-0663.94.4.660 CrossRefGoogle Scholar
  31. 31.
    Van Dinther, M., Dochy, F., Segers, M.: Factors affecting students’ self-efficacy in higher education. Educ. Res. Rev. 6, 95–108 (2011). doi: 10.1016/j.edurev.2010.10.003 CrossRefGoogle Scholar
  32. 32.
    Bandura, A.: Theoretical perspectives. In: Self-efficacy: The Exercise of Control, 604 pages. W.H. Freeman, New York (1997)Google Scholar
  33. 33.
    De Grez, L., Valcke, M., Roozen, I.: The impact of an innovative instructional intervention on the acquisition of oral presentation skills in higher education. Comput. Educ. 53, 112–120 (2009). doi: 10.1016/j.compedu.2009.01.005 CrossRefGoogle Scholar
  34. 34.
    Brand-Gruwel, S., Wopereis, I., Vermetten, Y.: Information problem solving by experts and novices: analysis of a complex cognitive skill. Comput. Hum. Behav. 21, 487–508 (2005). doi: 10.1016/j.chb.2004.10.005 CrossRefGoogle Scholar
  35. 35.
    De Grez, L., Valcke, M., Roozen, I.: The impact of goal orientation, self-reflection and personal characteristics on the acquisition of oral presentation skills. Eur. J. Psychol. Educ. 24, 293–306 (2009). doi: 10.1007/BF03174762 CrossRefGoogle Scholar
  36. 36.
    Mayer, R.E.: Multimedia Learning, 2nd edn. Cambridge University Press, New York (2009). doi: 10.1007/s13398-014-0173-7.2
  37. 37.
    Paivio, A.: Mental Representations: A Dual Coding Approach (2008). doi: 10.1093/acprof:oso/9780195066661.001.0001
  38. 38.
    Ayres, P.: State-of-the-art research into multimedia learning: a commentary on mayer’s handbook of multimedia learning. Appl. Cogn. Psychol. 29, 631–636 (2015). doi: 10.1002/acp.3142 CrossRefGoogle Scholar
  39. 39.
    Skulmowski, A., Pradel, S., Kühnert, T., et al.: Embodied learning using a tangible user interface: the effects of haptic perception and selective pointing on a spatial learning task. Comput. Educ. 92–93, 64–75 (2016). doi: 10.1016/j.compedu.2015.10.011 CrossRefGoogle Scholar
  40. 40.
    Moreno, R., Mayer, R.: Interactive multimodal learning environments. Educ. Psychol. Rev. 19, 309–326 (2007). doi: 10.1007/s10648-007-9047-2 CrossRefGoogle Scholar
  41. 41.
    Schweppe, J., Eitel, A., Rummer, R.: The multimedia effect and its stability over time. Learn. Instr. 38, 24–33 (2015). doi: 10.1016/j.learninstruc.2015.03.001 CrossRefGoogle Scholar
  42. 42.
    Eitel, A., Scheiter, K.: Picture or text first? Explaining sequence effects when learning with pictures and text. Educ. Psychol. Rev. 27, 153–180 (2014). doi: 10.1007/s10648-014-9264-4
  43. 43.
    Van Merriënboer, J.J.G., Kester, L.: The four-component instructional design model: multimedia principles in environment for complex learning. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, pp. 104–148. Cambridge University Press, New York (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Kevin Ackermans
    • 1
    Email author
  • Ellen Rusman
    • 1
  • Saskia Brand-Gruwel
    • 1
  • Marcus Specht
    • 1
  1. 1.Welten InstituteOpen UniversiteitHeerlenThe Netherlands

Personalised recommendations