Advances in Health Sciences Education

, Volume 10, Issue 3, pp 171–188

The Use of Observational Diaries in In-Training Evaluation: Student Perceptions

  • Marjan J.B. Govaerts
  • Cees P.M. van der Vleuten
  • Lambert W.T. Schuwirth
  • Arno M.M. Muijtjens
Article
  • 161 Downloads

Abstract

Introduction: In health science education clinical clerkships serve the twofold purpose of guiding student learning and assessment of performance. Evidently, both formative and summative assessment procedures are needed in clerkship assessment. In-training evaluation (ITE) has the potential to serve both assessment functions. Implementation of effective ITE, however, has been shown to be problematic, partly because integration of assessment functions may have negative consequences for teaching and learning. This study investigates student perceptions of the impact of an integrated assessment approach, seeking to refine criteria for effective ITE. Method: In the curriculum of Maastricht Midwifery School (MMS), clerkship assessment is based on ITE serving both assessment functions. The ITE model is based on principles of extensive work sampling, and frequent documentation of performance. A focus group technique was used to explore student perceptions on the impact of the ITE approach on student learning and supervisor teaching behaviour, and on the usefulness of information for decision making. Results: Results indicate that the assessment approach is effective in guidance of student learning. Furthermore, students consider the frequent performance documentation essential in clerkship grading. Acceptance and effectivity of ITE requires a learning environment which is safe and respectful. Transparency of assessment processes is the key to success. Suggestions for improvement focus on variation in evaluation formats, improvement of feedback (narrative, complete) and student involvement in assessment. Conclusion: ITE can fulfill both its formative and summative purposes when some crucial conditions are taken into account. Careful training of both supervisors and students in the use of ITE for student learning and performance measurement is essential.

Key words:

clerkship assessment in-training-evaluation performance assessment student perceptions 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Brinko, K.T. 1993The practice of giving feedback to improve teachingJournal of Higher Education64574593Google Scholar
  2. Cleveland, J.N., Murphy, K.R. 1992

    Analyzing performance appraisal as goal-directed behaviour

    Ferris, G.Rowland, K. eds. Research in Personnel and Human Resources ManagementJAI Press Greenwich, CT121185
    Google Scholar
  3. Cushing, A. 2002

    Assessment of non-cognitive factors.

    Norman, G.R.Vleuten, C.P.M.Newble, D.I. eds. International Handbook of Research in Medical EducationKluwer Academic PublishersDordrecht711755
    Google Scholar
  4. Ende, J. 1983Feedback in clinical medical educationJAMA250777781CrossRefPubMedGoogle Scholar
  5. Eraut, M. 1994Developing Professional Knowledge and CompetenceFalmer PressLondonGoogle Scholar
  6. Eva, K.W., Cunnington, J.P.W., Reiter, H.I., Keane, D.R., Norman, G.R. 2004How can I know what I don't know? Poor self assessment in a well-defined domainAdvances in Health Sciences Education9211224CrossRefPubMedGoogle Scholar
  7. Falchikov, N., Boud, D. 1989Student self-assessment in higher education: A meta-analysisReview of Educational Research59395430Google Scholar
  8. Feletti, G., Cameron, D., Dawson-Saunders, B., Groselliers, J-P., Farmer, E., McAvoy, P. 1994

    In-training assessment

    Newble, D.Jolly,  B.Wakeford, R. eds. The Certification and Recertification of DoctorsCambridge University PressCambridge151166
    Google Scholar
  9. Gielen, S., Dochy, F., Dierick, S. 2003

    Evaluating the consequential validity of new modes of assessment: The influence of assessment on learning, including pre-, post-, and true assessment effects

    Segers, M.Dochy, F.Cascallar, E. eds. Optimising New Modes of Assessment: In Search of Qualities and StandardsKluwer Academic PublishersDordrecht3754
    Google Scholar
  10. Gordon, M.J. 1991A review of the validity and accuracy of self-assessments in health professions trainingAcademic Medicine66762769PubMedGoogle Scholar
  11. Gordon, M.J. 1992Self-assessment programmes and their implications for health professions trainingAcademic Medicine67672679PubMedGoogle Scholar
  12. Govaerts, M.J.B., Vleuten, C.P.M., Schuwirth, L.W.T. 2002Optimising the reproducibility of a performance-based test in midwiferyAdvances in Health Sciences Education7133145CrossRefPubMedGoogle Scholar
  13. Gray, J.D. 1996Global rating scales in residency educationAcademic Medicine21S5563Google Scholar
  14. Handfield-Jones, R.S., Mann, K.V., Challis, M.E., Hobma, S.O., Klass, D.J., McManus, I.C., Paget, N.S.,  et al. 2002Linking assessment to learning: A new route to quality assurance in medical practiceMedical Education36949958CrossRefPubMedGoogle Scholar
  15. Hays, R., Wellard, R. 1998In-training assessment in postgraduate training for general practiceMedical Education32507513CrossRefPubMedGoogle Scholar
  16. Hays, R.B., Jolly, B.C., Caldon, L.J.M., McCrorie, P., McAvoy, P.A., McManus, I.C.,  et al. 2002Is insight important? Measuring capacity to change performanceMedical Education36965971CrossRefPubMedGoogle Scholar
  17. Hodges, B., Regehr, G., McNaughton, N., Tiberius, R., Hanson, M. 1999OSCE checklists do not capture increasing levels of expertiseAcademic Medicine7411291134PubMedGoogle Scholar
  18. Imseis, H.M., Galvin, S.L. 2004Faculty and resident preference for two different forms of lecture evaluationAmerican Journal of Obstetrics and Gynecology19118151821CrossRefPubMedGoogle Scholar
  19. Irby, D.M. 1995Teaching and learning in ambulatory care settings: A thematic review of the literatureAcademic Medicine70898931PubMedGoogle Scholar
  20. Kilminster, S.M., Jolly, B.C. 2000Effective supervision in clinical practice settings: A literature reviewMedical Education34827840CrossRefPubMedGoogle Scholar
  21. McIlroy, J.H., Hodges, B., McNaughton, N., Regehr, G. 2002The effect of candidates' perceptions of the evaluation method on reliability of checklist and global rating scores in an objective structured clinical examinationAcademic Medicine77725728PubMedGoogle Scholar
  22. Miller, G.E. 1990The assessment of clinical skills/competence/performanceAcademic Medicine65S6367PubMedGoogle Scholar
  23. Morgan, D. (1997). Focus Groups as Qualitative Research. Thousand Oaks etc: Sage Publications.Google Scholar
  24. Norman, G.R., Vleuten, C.P.M., Graaf, E. 1991Pitfalls in the pursuit of objectivity: Issues of validity, efficiency, and acceptabilityMedical Education25119126PubMedGoogle Scholar
  25. Regehr, G., MacRae, H., Reznick, R.K., Szalay, D. 1998Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examinationAcademic Medicine73993997PubMedGoogle Scholar
  26. Rolfe, I., McPherson, J. 1995Formative assessment: How am I doing?The Lancet345837839CrossRefGoogle Scholar
  27. Rothman, A.I., Blackmore, D., Daupinee, W.D., Reznick, R. 1997The use of global ratings in OSCE station scoresAdvances in Health Sciences Education1215219CrossRefGoogle Scholar
  28. Sommers, P.S., Muller, J.H., Saba, G.W., Draisin, J.A., Shore, W.B. 1994Reflections-on-action: Medical students' accounts of their implicit beliefs and strategies in the context of one-to-one clinical teachingAcademic Medicine69S846PubMedGoogle Scholar
  29. Spike, N., Alexander, H., Elliott, S., Hazlett, C., Kilminster, S., Prideaux, D., Roberts, T. 2000In-training assessment – its potential in enhancing clinical teachingMedical Education3485861CrossRefPubMedGoogle Scholar
  30. Stewart, J., O’Halloran, C., Barton, J.R., Singleton, S.J., Harrigan, P., Spencer, J. 2000Clarifying the concepts of confidence and competence to produce appropriate self-evaluation measurement scalesMedical Education34903909CrossRefPubMedGoogle Scholar
  31. Toohey, S., Ryan, G., Hughes, C. 1996Assessing the practicumAssessment and Evaluation in Higher Education21215227Google Scholar
  32. Topping, K. 2003

    Self and peer assessment in school and university: Reliability, validity and utility

    Segers, M.Dochy, F.Cascallar, E. eds. Optimising New Modes of Assessment: In Search of Qualities and StandardsKluwer Academic PublishersDordrecht5589
    Google Scholar
  33. Turnbull, J., Barneveld, C. 2002

    Assessment of clinical performance: In-training evaluation

    Norman, G.R.Vleuten, C.P.M.Newble, D.I. eds. International Handbook of Research in Medical EducationKluwer Academic PublishersDordrecht793810
    Google Scholar
  34. Van Luijk, S.J., Van der Vleuten, C.P.M., Schelven, R.M. 1990

    The relation between content and psychometric characteristics in performance-based testing

    Bender, W.Hiemstra, R.J.Scherpbier, A.J.J.A.Zwierstra, R.P. eds. Teaching and Assessing Clinical CompetenceBoekwerk PublicationsGroningen202207
    Google Scholar
  35. Vleuten, C.P.M., Norman, G.R., Graaf, E. 1991Pitfalls in the pursuit of objectivity: Issues of reliabilityMedical Education25110118PubMedGoogle Scholar
  36. Vleuten, C.P.M., Scherpbier, A.J.J.A., Dolmans, D.H.J.M., Schuwirth, L.W.T., Verwijnen, G.M., Wolfhagen, H.A.P. 2000Clerkship assessment assessedMedical Teacher22592600CrossRefGoogle Scholar

Copyright information

© Springer 2005

Authors and Affiliations

  • Marjan J.B. Govaerts
    • 1
  • Cees P.M. van der Vleuten
    • 2
  • Lambert W.T. Schuwirth
    • 2
  • Arno M.M. Muijtjens
    • 2
  1. 1.Opleiding tot VerloskundigeVroedvrouwenschoolMaastrichtThe Netherlands
  2. 2.Department of Educational Development and ResearchMaastricht UniversityThe Netherlands

Personalised recommendations