Skip to main content
SpringerLink
Log in

The Modern Surgeon and Competency Assessment: Are the Workplace-Based Assessments Evidence-Based?

  • Original Scientific Report
  • Published:
World Journal of Surgery Aims and scope Submit manuscript

Abstract

Introduction

The assessment of higher surgical training has changed in the last decade or two, with a greater emphasis on work-based assessments (WBAs) to prove competency. The aim of this study was to determine the evidence underpinning the use and number of WBAs in surgical training.

Methods

In July 2013, a systematic electronic literature review was undertaken using PubMed (Medline), Embase, Google Scholar and the Cochrane library.

Results

A total of 27 studies met the inclusion criteria of which 25 were observational studies and only five assessed WBAs in a surgical setting. Validity and feasibility in surgical training were assessed in two studies, respectively, with the results suggesting that WBAs maybe neither feasible nor valid in surgical training. The number required to achieve reliability in surgical training was demonstrated to be three in two separate studies. The evidence for the reliability, feasibility and validity of WBAs in other non-surgical fields was conflicting.

Conclusion

There is a paucity of evidence supporting the use of WBAs as a tool to determine competency in surgical training, and as such, they should only have a limited role in training until more evidence is available. There appears to be no justification or evidence underpinning the use of a specific number of WBAs to determine surgical competency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Fitzgerald JEF, Giddings CEB, Khera G et al (2012) Improving the future of surgical training and education: consensus recommendations from the Association of Surgeons in Training. Int J Surg 10(8):389–392

    Article  CAS  PubMed  Google Scholar 

  2. Shalhoub J, Vesey AT, Fitzgerald JE (2014) What evidence is there for the use of Workplace-based assessment in surgical training? J Surg Educ. doi:10.1016/j.jsurg.2014.03.013

  3. General Medical Council (2011) Learning and assessment in the clinical environment: the way forward. General Medical Council, United Kingdom

  4. Munsch C (2009) British surgeons’ experiences of mandatory online workplace-based assessment—reply to authors. J R Soc Med 102(9):357–358

    Article  PubMed Central  PubMed  Google Scholar 

  5. Pentlow A (2013) Workplace-based assessments in surgery: are we heading in the wrong direction? Med Teach 35(1):80

    Article  PubMed  Google Scholar 

  6. Bindal T, Wall D, Goodyear HM (2011) Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise? Med Teach 33(11):919–927

    Article  PubMed  Google Scholar 

  7. Moher D, Liberati A, Tetzlaff J et al (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 339:b2535

    Article  PubMed Central  PubMed  Google Scholar 

  8. Eraut M (2009) The ISCP evaluation report. Joint Committee on Surgical Training

  9. ISCP Management Group (2012) ISCP evaluation report. http://www.jcst.org/ISCP%20Evaluation%20/full_evaluation_report

  10. Buckley S, Coleman J, Davison I et al (2009) The educational effects of portfolios on undergraduate student learning: a best evidence medical education (BEME) systematic review. BEME Guide No 11. Med Teach 31:282–298

    Article  PubMed  Google Scholar 

  11. Barton JR, Corbett S, van der Vleuten CP (2012) The validity and reliability of a direct observation of procedural skills assessment tool: assessing colonoscopic skills of senior endoscopists. Gastrointest Endosc 75(3):591–597

    Article  PubMed  Google Scholar 

  12. Brazil V, Ratcliffe L, Zhang J et al (2012) Mini-CEX as a workplace-based assessment tool for interns in an emergency department—does cost outweight value? Med Teach 34:1017–1023

    Article  PubMed  Google Scholar 

  13. Beard JD (2008) Assessment of surgical skills of trainees in the UK. Ann R Coll Surg Engl 90:282–285

    Article  PubMed Central  PubMed  Google Scholar 

  14. Crossley J, Johnson G, Booth J et al (2011) Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ 45:560–569

    Article  PubMed  Google Scholar 

  15. De Lima AA, Barrero C, Bararra S et al (2007) Validity, reliability, feasibility and satisfaction of the mini-clinical evaluation exercise (Mini-CEX) for cardiology residency training. Med Teach 29:785–790

    Article  Google Scholar 

  16. Durning SJ, Cation LJ, Markert RJ (2002) Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med 77(9):900–904

    Article  PubMed  Google Scholar 

  17. Hassan S (2011) Faculty development: mini-CEX as workplace-based assessment. Educ Med J 3(1):12–21

    Article  Google Scholar 

  18. Hill F, Kendall K, Galbraith K et al (2009) Implementing the undergraduate mini-CEX: a tailored approach at Southampton University. Med Educ 43:326–334

    Article  PubMed  Google Scholar 

  19. Holmboe ES, Huot S, Chung J et al (2003) Construct validity of the mini-clinical evaluation exercise (minCEX). Acad Med 78(8):826–830

    Article  PubMed  Google Scholar 

  20. James K, Cross K, Lucarotti ME et al (2009) Undertaking procedure-based assessment is feasible in clinical practice. Ann R Coll Surg Engl 91:110–112

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  21. Johnson G, Booth J, Crossley J et al (2011) Assessing trainees in the workplace: results of a pilot study. Clin Med 11(1):48–53

    Article  PubMed  Google Scholar 

  22. Kogan JR, Bellini LM, Shea JA (2003) Feasibility, reliability and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 78(10):S33–S35

    Article  PubMed  Google Scholar 

  23. Kogan JR, Bellini LM, Shea JA (2002) Implementation of the mini-CEX to evaluate medical students’ clinical skills. Acad Med 77(11):1156–1157

    Article  PubMed  Google Scholar 

  24. Malhotra S, Hatalal R, Courneya CA (2008) Internal medicine residents’ perceptions of the mini-clinical evaluation exercise. Med Teach 30:414–419

    Article  PubMed  Google Scholar 

  25. Marriott J, Purdie H, Crossley J et al (2011) Evaluation of procedure-based assessment for assessing trainees’ skills in the operating theatre. Br J Surg 98:450–457

    Article  CAS  PubMed  Google Scholar 

  26. Mitchell C, Bhat S, Herbert A et al (2011) Workplace-based assessments of junior doctors: do scores predict training difficulties? Med Educ 45:1190–1198

    Article  PubMed  Google Scholar 

  27. Moonen-van Loon JMW, Overeem K, Donkers HHLM et al (2013) Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ 18(5):1087–1102

    Article  CAS  Google Scholar 

  28. Morris A, Hewitt J, Roberts CM (2006) Practical experience of using directly observed procedures, mini clinical evaluation examinations, and peer observation in pre-registration house officer (FY1) trainees. Postgrad Med J 82:285–288

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  29. Nair B, Hensley MJ, Parvathy MS et al (2012) A systematic approach to workplace-based assessment for international medical graduates. Med J Aust 196:399–402

    Article  PubMed  Google Scholar 

  30. Nair BR, Alexander HG, McGrath BP et al (2008) The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates. Med Educ 189(3):159–161

    Google Scholar 

  31. Norcini JJ, Blank LL, Arnold GK et al (1995) The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med 123:795–799

    Article  CAS  PubMed  Google Scholar 

  32. Norcini JJ, Blank LL, Arnold GK et al (1997) Examiner differences in the mini-Cex. Adv Health Sci Educ 2:27–33

    Article  CAS  Google Scholar 

  33. Norcini JJ, Blank LL, Duffy DF et al (2003) The mini-CEX: a method for assessing clinical skills. Ann Intern Med 138:476–481

    Article  PubMed  Google Scholar 

  34. Playford D, Kirke A, Maley M et al (2013) Longitudinal assessment in an undergraduate longitudinal integrated clerkship: the mini clinical evaluation exercise (mCEX) profile. Med Teach 35(8):1412–1421

    Article  Google Scholar 

  35. Satpathy A, Kneebone R (2011) Workplace-based assessment in surgical training: a qualitative study. Ann R Coll Surg Engl 93(5):1–5

    Article  Google Scholar 

  36. Wilkinson JR, Crossley JGM, Wragg A et al (2008) Implementing workplace-based assessment across the medical specialties in the United Kingdom. Med Educ 42:364–373

    Article  PubMed  Google Scholar 

  37. Weller JM, Jolly B, Misur MP et al (2009) Mini-clinical evaluation exercise in anaesthesia training. Br J Anaesth 102:633–641

    Article  CAS  PubMed  Google Scholar 

  38. Beard J, Rowley D, Bussey M et al (2009) Workplace-based assessment: assessing technical skill throughout the continuum of surgical training. ANZ J Surg 79(3):148–153

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. West Middlesex Hospital, Twickenham Rd, London, Middlesex, TW7 6AF, UK

    K. M. Torsney

  2. Division of Surgery and Cancer, Chelsea and Westminster Campus, Imperial College, London, UK

    D. M. Cocker & A. A. P. Slesser

Authors
  1. K. M. Torsney
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. M. Cocker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. A. P. Slesser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. M. Torsney.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Torsney, K.M., Cocker, D.M. & Slesser, A.A.P. The Modern Surgeon and Competency Assessment: Are the Workplace-Based Assessments Evidence-Based?. World J Surg 39, 623–633 (2015). https://doi.org/10.1007/s00268-014-2875-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00268-014-2875-6

Keywords

Search

Navigation