Skip to main content
SpringerLink
Log in

Computer-assessed performance of psychomotor skills in endoscopic otolaryngology surgery: construct validity of the Dundee Endoscopic Psychomotor Otolaryngology Surgery Trainer (DEPOST)

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

This study was undertaken to introduce and establish the value of the Dundee Endoscopic Psychomotor Otolaryngology Surgery Trainer (DEPOST) as a customisable, objective real-time scoring system for trainee assessment. The construct validity of the system was assessed by comparing the performance of experienced otolaryngologists with that of otolaryngology trainees, junior doctors and medical students.

Methods

Forty two subjects (13 Consultants, 8 senior trainees, 13 junior trainees and 8 junior doctors/medical students) completed a single test on DEPOST. The test involved using a 30° rigid endoscope and a probe with position sensor, to identify a series of lights in a complex 3-dimensional model. The system scored subjects for time, success rate, and economy of movement (distance travelled). An analysis of variance and correlation analysis were used for the data analysis, with statistical significance set at 0.05.

Results

Increasing experience led to significantly improved performance with the DEPOST (p < 0.01). Senior trainees’ results were significantly better than those of consultant otolaryngologists in success rate and time (p < 0.05 & p < 0.05). Consultants were the most efficient in their movement (p = 0.051)

Conclusions

The system provides an accurate and customisable assessment of endoscopic skill in otolaryngologists. The DEPOST system has construct validity, with master surgeons and senior trainees completing the tasks more accurately without sacrificing execution time, success rate or efficiency of movement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165:358–361

    Article  CAS  PubMed  Google Scholar 

  2. Darzi A, Smith S, Taffinder N (1999) Assessing operative skill. Needs to become more objective. BMJ 318:887–888

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  3. McGreevy JM (2005) The aviation paradigm and surgical education. J Am Coll Surg 201:110–117

    Article  PubMed  Google Scholar 

  4. Tansley P, Kakar S, Withey S, Butler P (2007) Visuospatial and technical ability in the selection and assessment of higher surgical trainees in the London Deanery. Ann R Coll Surg Engl 89:591–595

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  5. Frances NK, Hanna GB, Cuschieri A (2001) Reliability of the advanced dundee endoscopic psychomotor tester for bimanual tasks. Arch Surg 136(1):40–44

    Article  Google Scholar 

  6. Arora H, Uribe J, Ralph W, Zeltsan M, Cuellar H, Gallagher A, Fried MP (2005) Assessment of construct validity of the endoscopic sinus surgery simulator. Arch Otolaryngol Head Neck Surg 131(3):217–221

    Article  PubMed  Google Scholar 

  7. Weghorst S, Airola C, Oppenheimer P, Edmond CV, Patience T, Heskamp D, Miller J (1998) Validation of the madigan ESS simulator. Stud Health Technol Inform 50:399–405

    CAS  PubMed  Google Scholar 

  8. Hanna GB, Drew T, Clinch P, Hunter B, Shimi S, Dunkley MP, Cuschieri A (1996) A microrpocessor-controlled psychomotor tester for minimal access surgery. Surg Endoscopy 10:965–969

    Article  CAS  Google Scholar 

  9. Francis NK, Hanna GB, Cuschieri A (2002) The performance of master surgeons on the advanced dundee endoscopic psychomotor tester. Contrast validity study. Arch Surg 137(7):841–844

    Article  PubMed  Google Scholar 

  10. Macmillan AIM, Cuschieri A (1999) Assessment of innate ability and skills for endoscopic manipulations by the Advanced Dundee Endoscopic Psychomotor Tester: predictive and concurrent validity. Am J Surg 177(3):274–277

    Article  CAS  PubMed  Google Scholar 

  11. Schijven MP, Jakimowicz J, Schot C (2002) The Advanced Dundee Endoscopic Psychomotor Tester (ADEPT) objectifying subjective psychomotor test performance. Surg Endosc 16(6):943–948

    Article  CAS  PubMed  Google Scholar 

  12. Breatnach E, Abbott GC, Fraser RG (1984) Dimensions of the normal human trachea. Am J Roentgenol 141:903–906

    Article  Google Scholar 

  13. Meskers CGM, Fraterman H, van der Helm FC, Vermeulen HM, Rozing PM (1999) Calibration of the “Flock of Birds’’ electromagnetic tracking device. J Biomech 32:629–633

    Article  CAS  PubMed  Google Scholar 

  14. Ramsey PG, Carline JD, Inui TS, Larson EB, LoGerfo JP, Norcini JJ, Wenrich MD (1991) Changes over time in the knowledge base of practicing internists. JAMA 266:1103–1107

    Article  CAS  PubMed  Google Scholar 

  15. Stolley PD, Becker MH, Lasagna L, McEvilla JD, Sloane LM (1972) The relationship between physician characteristics and prescribing appropriateness. Med Care 10:17–28

    Article  CAS  PubMed  Google Scholar 

  16. Waljee JF, Greenfield LJ, Dimick JB, Birkmeyer JD (2006) Surgeon age and operative mortality in the United States. Ann Surg 244(3):353–362

    PubMed Central  PubMed  Google Scholar 

  17. Duclos A, Peix J, Colin C, Kraimps J, Menegaux F, Pattou F (2012) Influence of experience on performance of individual surgeons in thyroid surgery: prospective cross sectional multicentre study. BMJ 34:344

    Google Scholar 

  18. Abel EW, Zhuo Y, Ross PD, White PS (2013) Automatic glare removal in endoscopic imaging. Surg Endosc 28(2):584–591

    Article  PubMed  Google Scholar 

  19. White PS, Nassif R, Saleh H, Drew T (2004) Pilot study of a device for measuring instrument forces during endoscopic sinus surgery. Acta Otolaryngol 124(2):176–178

    Article  PubMed  Google Scholar 

  20. Joice P, Ross PD, Wang D, Abel EW, White PS (2012) Measurement of osteotomy force during endoscopic sinus surgery. Allergy Rhinol (Providence) 3(2):e61–e65

    Article  Google Scholar 

  21. Ross PD, Joice P, Nassif RG, White PS (2010) Surgical instrument force exerted during endoscopic sinus surgery: differences within the ethmoid sinus complex. Am J Rhinol Allergy 24(1):76–80

    Article  PubMed  Google Scholar 

  22. Sheedy MB, Bergin M, Wylie G, Ross P, Dove R, Bird P (2012) Development of a surgical instrument for measuring forces applied to the ossicles of the middle ear. Aust Phys Eng Sci Med 35(4):503–510

    Article  Google Scholar 

Download references

Acknowledgments

The authors wish to thank those who participated in the study, including the members of ENT Scotland.

Disclosures

Peter D. Ross, Richard Steven, Dong Zhang, Heng Li and Eric W. Abel have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Department of Otolaryngology, Ninewells Hospital and Medical School, University of Dundee, Dundee, DD1 9SY, Scotland, UK

    Peter D. Ross & Richard Steven

  2. School of Engineering, University of Dundee, Dundee, DD1 4HN, Scotland, UK

    Dong Zhang, Heng Li & Eric W. Abel

Authors
  1. Peter D. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Steven
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Heng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eric W. Abel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter D. Ross.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ross, P.D., Steven, R., Zhang, D. et al. Computer-assessed performance of psychomotor skills in endoscopic otolaryngology surgery: construct validity of the Dundee Endoscopic Psychomotor Otolaryngology Surgery Trainer (DEPOST). Surg Endosc 29, 3125–3131 (2015). https://doi.org/10.1007/s00464-014-4036-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-014-4036-2

Keywords

Search

Navigation