Skip to main content
SpringerLink
Log in
Book cover

Handbook of Vocational Education and Training pp 1–20Cite as

Electronic Portfolios Enhanced with Learning Analytics at the Workplace

  • Living reference work entry
  • First Online:

Abstract

During workplace-based learning, e.g., clinical or during an internship, supervisors’ quality of feedback and assessment is crucial for trainees’ expertise development. Electronic portfolios (ePortfolios) are often used as tools for longitudinal assessment at the workplace. So far, ePortfolios have not realized their full potential, since they are often not well tailored to the workplace and trainees’ needs. Potential data about trainees’ behavior at the workplace is generally underused, and the management of the data is complex. It is expected that ePortfolios enhanced with learning analytics may serve as means to improve the quality of feedback and assessment regarding trainees’ progress and development. This chapter addresses this by outlining an approach that is applied in a European 7th framework project (www.project-WatchMe.eu).

This chapter shows the development of an ePortfolio environment enhanced with learning analytics, to be used at the workplace in medical, veterinary, and teacher education. Evaluation took place by means of a quasi-experimental design regarding the impact of this environment on trainees’ motivation, their assessment experience, and their use. Data gathered in four institutes for medical, veterinary, and teacher education (n = 217) showed that trainees were highly motivated for their internships and positively evaluated the perceived feedback. The use of learning analytics features varied. In general visual feedback by means of a timeline of trainees’ progress was mostly used, while trainees barely used the features with written feedback. It is concluded that the promise of learning analytics connected to ePortfolios can only be fulfilled when developed and implemented through the eyes of the users.

This is a preview of subscription content, log in via an institution.

References

  • Agudo-Peregrina A, Iglesias-Pradas S, Conde-González MA, Hernández-García A (2014) Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning. Comput Hum Behav 31:542–550. https://doi.org/10.1016/j.chb.2013.05.031

    Article  Google Scholar 

  • Barbera E (2009) Mutual feedback in e-portfolio assessment: an approach to the netfolio system. Br J Educ Technol 40(2):342–357. https://doi.org/10.1111/j.1467-8535.2007.00803.x

    Article  Google Scholar 

  • Barrett HC (1998) Strategic questions: what to consider when planning for electronic portfolios. Learn Lead Technol 26:6–13

    Google Scholar 

  • Billett S (2004) Workplace participatory practices: conceptualising workplaces as learning environments. J Work Learn 16(6):312–324. https://doi.org/10.1108/13665620410550295

    Article  Google Scholar 

  • Bryant LH, Chittum JR (2013) ePortfolio effectiveness: a(n ill-fated) search for empirical support. Int J ePortfolio 3(2):189–198

    Google Scholar 

  • Butler P (2006) A review of the literature on portfolios and electronic portfolios. Massey University College of Education, Palmerston North

    Google Scholar 

  • Dekker-Groen A, Van der Schaaf M, Stokking K (2012) Performance standards for teachers supporting nursing students’ reflection skills development. J Nurs Educ Pract 2(1):9–19. https://doi.org/10.5430/jnep.vn1p9

    Article  Google Scholar 

  • Doyle GJ, Garrett B, Currie LM (2014) Integrating mobile devices into nursing curricula: opportunities for implementation using Rogers’ diffusion of innovation model. Nurse Educ Today 34(5):775–782. https://doi.org/10.1016/j.nedt.2013.10.021

    Article  Google Scholar 

  • Duijn CC, Welink LS, Mandoki M, ten Cate OT, Kremer WD, Bok HG (2017) Am I ready for it? Students’ perceptions of meaningful feedback on entrustable professional activities. Perspect Med Educ 6:256. https://doi.org/10.1007/s40037-017-0361-1

    Article  Google Scholar 

  • Dysthe O, Engelsen KS (2011) Portfolio practices in higher education in Norway in an international perspective: macro-, meso-and micro-level influences. Assess Eval High Educ 36(1):63–79. https://doi.org/10.1080/02602930903197891

    Article  Google Scholar 

  • Elias T (2011) Learning analytics: definitions, processes and potential. Creative Commons Attribution 3.0

    Google Scholar 

  • Ericsson A, Pool R (2016) Peak: secrets from the new science of expertise. Harcourt, Houghton Mifflin

    Google Scholar 

  • Ericsson KA, Charness N, Feltovich PJ, Hoffman RR (eds) (2006) The Cambridge handbook of expertise and expert performance. Cambridge University Press, New York

    Google Scholar 

  • Eynon B, Gambino LM, Török J (2014) What difference can ePortfolio make? A field report from the connect to learning project. Int J ePortfolio 4(1):95–114

    Google Scholar 

  • Feltovich PJ, Prietula MJ, Ericsson KA (2006) Studies of expertise from psychological perspectives. In: Ericsson KA, Charness N, Feltovich P, Hoffman RR (eds) Cambridge handbook of expertise and expert performance. Cambridge University Press, Cambridge, pp 41–67

    Chapter  Google Scholar 

  • Frank JR (2005) The CanMEDS 2005 physician competency framework. Better standards, better physicians. Better care, Ottawa, The Royal College of Physicians and Surgeons of Canada

    Google Scholar 

  • Gibbs G, Simpson C (2003) Measuring the response of students to assessment: the assessment experience questionnaire. In: 11th Improving Student Learning Symposium, pp 1–12

    Google Scholar 

  • Greller W, Drachsler H (2012) Translating learning into numbers: a generic framework for learning analytics. J Educ Technol Soc 15(3):42–57

    Google Scholar 

  • Grossman P, Hammerness K, McDonald M (2009) Redefining teaching, re-imagining teacher education. Teach Teach Theory Pract 15(2):273–289. https://doi.org/10.1080/13540600902875340

    Article  Google Scholar 

  • Hamp-Lyons L, Condon W (2000) Assessing the portfolio: principles for practice, theory, and research. Hampton Press, Cresskill

    Google Scholar 

  • Hattie J, Timperley H (2007) The power of feedback. Rev Educ Res 77(1):81–112. https://doi.org/10.3102/003465430298487

    Article  Google Scholar 

  • Jafari A, Kaufman C (eds) (2006) Handbook of research on eportfolios. IDEA Group Publishing, Hershey

    Google Scholar 

  • Jarafsky D, Martin JH (2008) Speech and language processing: an introduction to natural language processing, computational linguistics and speech recognition, 2nd edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Jonker G, Hoff RG, Ten Cate O (2015) A case for competency-based anaesthesiology training with entrustable professional activities: an agenda for development and research. Eur J Anaesthesiol (EJA) 32(2):71–76. https://doi.org/10.1097/EJA.0000000000000109

    Article  Google Scholar 

  • Kane MT (1992) An argument-based approach to validity. Psychol Bull 112(3):527

    Article  Google Scholar 

  • Kennelly E, Osborn D, Reardon R, Shetty B (2016) Guidance for ePortfolio researchers: a case study with implications for the ePortfolio domain. Int J ePortfolio 6(2):117–125. http://www.theijep.com

    Google Scholar 

  • Kovanović V, Joksimović S, Gašević D, Hatala M, Siemens G (2015) Content analytics: the definition, scope, and an overview of published research. In: Handbook of learning analytics. SoLAR, Edmonton. https://doi.org/10.18608/hla17.007

    Chapter  Google Scholar 

  • Laskey KB (2008) MEBN: a language for first-order Bayesian knowledge bases. Artif Intell 172(2–3):140–178. https://doi.org/10.1016/j.artint.2007.09.006

    Article  Google Scholar 

  • Leijen A, Slof B, Malva L, Hunt P, van Tartwijk JWF, van der Schaaf MF (2017) Performance-based competency requirements for learner teachers and how to assess them. Int J Inf Educ Technol 7(3):190–194. https://doi.org/10.18178/ijiet.2017.7.3.864

    Article  Google Scholar 

  • McNamara DS, Allen LK, Crossley SA, Dascula M, Perret CA (2017) Natural language processing and learning analytics. In: Lang C, Siemens G, Wise A, Gašević D (eds) Handbook of learning analytics. SoLAR, Edmonton, pp 93–104. https://doi.org/10.18608/hla17.008

    Chapter  Google Scholar 

  • Mislevy RJ, Behrens JT, Dicerbo KE, Levy R (2012) Design and discovery in educational assessment: evidence-centered design, psychometrics, and educational data mining. JEDM-J Educ Data Min 4(1):11–48

    Google Scholar 

  • Mulder M (2014) Conceptions of professional competence. In: Billett S, Harteis C, Gruber H (eds) International handbook of research in professional and practice-based learning. Springer, Dordrecht, pp 107–137

    Google Scholar 

  • Peacock S, Murray S, Scott A, Kelly J (2011) The transformative role of ePortfolios: feedback in healthcare learning. Int J ePortfolio 1(1):33–48. http://www.theijep.com

    Google Scholar 

  • Rezgui K, Mhiri H, Ghédira K (2014) Ontology-based e-Portfolio modeling for supporting lifelong competency assessment and development. Procedia Comput Sci 112:397–406. https://doi.org/10.1016/j.proc.2017.08l041

    Article  Google Scholar 

  • Royal College of Veterinary Surgeons (RCVS) (2014) Practice standards scheme manual. Royal College of Veterinary Surgeons, London

    Google Scholar 

  • Ryan RM, Deci EL (2000) Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 55(1):68. https://doi.org/10.1037/110003-066X.55.1.68

    Article  Google Scholar 

  • Sadler DR (2010) Beyond feedback: developing student capability in complex appraisal. Assess Eval High Educ 35(5):535–550. https://doi.org/10.1080/02602930903541015

    Article  Google Scholar 

  • Selwyn N (2015) Data entry: towards the critical study of digital data and education. Learn Media Technol 40(1):64–82. https://doi.org/10.1080/17439884.2014.921628

    Article  Google Scholar 

  • Siemens G, Gašević D (2012) Guest editorial – learning and knowledge analytics. Educ Technol Soc 15(3):1–2

    Google Scholar 

  • Ten Cate O (2005) Entrustability of professional activities and competency-bases training. Med Educ 39:1176–1177

    Article  Google Scholar 

  • Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, Van der Schaaf M (2015) Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no 99. Med Teach 37(11):983–1002

    Article  Google Scholar 

  • Toulmin SE (2003) The uses of argument. Updated edition. Cambridge University Press, Cambridge, UK

    Google Scholar 

  • Van der Schaaf MF, Stokking K, Verloop N (2008) Developing and validating a design for teacher portfolio assessment. Assess Eval High Educ 33(3):245–262. https://doi.org/10.1080/02602930701292522

    Article  Google Scholar 

  • Van der Schaaf M, Baartman L, Prins F (2012) Exploring the role of assessment criteria during teachers’ collaborative judgement processes of students’ portfolios. Assess Eval High Educ 37(7):847–860

    Article  Google Scholar 

  • van der Schaaf M, Donkers J, Slof B, Moonen-van Loon J, van Tartwijk J, Driessen E, ... Ten Cate O (2017) Improving workplace-based assessment and feedback by an E-portfolio enhanced with learning analytics. Educ Technol Res Dev 65(2):359–380. https://doi.org/10.1007/s11423-016-9496-8

  • Van Schaik S, Plant J, O’sullivan P (2013) Promoting self-directed learning through portfolios in undergraduate medical education: the mentors’ perspective. Med Teach 35(2):139–144

    Article  Google Scholar 

  • Warren SJ, Lee J, Najmi A (2014) The impact of technology and theory on instructional design since 2000. In: Spector JM, Merrill MD, Elen\ J, Bishop MJ (eds) Handbook of research on educational communications and technology, 4th edn. Springer, New York, pp 89–99

    Chapter  Google Scholar 

  • Wisman-Zwarter N, Van der Schaaf M, Ten Cate OT, Jonker G, Van Klei WA, Hoff R (2016) Defining the content of Anaesthesiology training with entrustable professional activities. A delphi study. Eur J Anesthesiol 33(8):559–567

    Article  Google Scholar 

Download references

Acknowledgment

This study was conducted within the framework of the “Workplace-Based e-Assessment Technology for competency-Based Higher Multi-Professional Education” (WATCHME) project, supported by the European Commission 7th Framework Programme (grant agreement No. 619349). We thank all participants and members of the WATCHME project for their contributions: Charité Universitätsmedizin Berlin, Germany; Jayway, Denmark; Maastricht University, the Netherlands; Mateum BV., the Netherlands; NetRom Software SRL, Romania; Tartu Ülikool, Estonia; University of California San Francisco, USA; University Medical Center Utrecht, the Netherlands; University of Reading, UK; University of Veterinary Medicine, Hungary; and Utrecht University, the Netherlands.

Author information

Authors and Affiliations

  1. University Medical Centre Utrecht and Utrecht University, Utrecht, The Netherlands

    M. F. van der Schaaf

  2. Center for Education and Department of Education, Utrecht University, Utrecht, The Netherlands

    M. F. van der Schaaf

Authors
  1. M. F. van der Schaaf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. F. van der Schaaf .

Editor information

Editors and Affiliations

  1. School of Education, University of Nottingham School of Education, Nottingham, UK

    Simon McGrath

  2. Education and Competence Studies Group, Wageningen University, Wageningen, Gelderland, The Netherlands

    Martin Mulder

  3. Faculty of Education, University of the Western Cape Faculty of Education, Bellville, South Africa

    Joy Papier

  4. School of Education, University of Nottingham, Nottingham, UK

    Rebecca Suart

Section Editor information

  1. Vocational Education and Training, University of Duisburg-Essen, Duisburg, Germany

    Esther Winther

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

van der Schaaf, M.F. (2019). Electronic Portfolios Enhanced with Learning Analytics at the Workplace. In: McGrath, S., Mulder, M., Papier, J., Suart, R. (eds) Handbook of Vocational Education and Training. Springer, Cham. https://doi.org/10.1007/978-3-319-49789-1_89-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49789-1_89-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49789-1

  • Online ISBN: 978-3-319-49789-1

  • eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education

Publish with us

Policies and ethics

Search

Navigation