Skip to main content
SpringerLink
Log in

Assessment of colorectal polyp recognition skill: development and validation of an objective test

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The quality of colonoscopy is known to vary. The extent to which colonoscopists can recognize the presence of subtle colorectal lesions by visually distinguishing them from the surrounding mucosa (i.e., polyp recognition skill) may be one of several attributes that influence polyp detection rates. The aim of the present study was to develop and validate the first objective test of polyp recognition skill.

Methods

Validation study. Twenty-eight experienced colonoscopists and eighty novices took a preliminary 280-item computer-based polyp recognition test. Items were genuine endoscopic images which participants assessed for the presence of “likely polyps.” Half included clinically identified polyps. Participants clicked on a suspected lesion or a button marked “no likely polyp”, and the main outcome measures were accuracy and response latency. The best items were selected for the final 50-item test.

Results

In the preliminary test, experienced colonoscopists correctly identified more polyps than novices (P < .0001) and better discriminated between clinically identified polyps and non-polyp features (as measured by d′, P < .0001). For polyp items, the experienced group also responded faster (P < .01). Effect sizes were large for accuracy (Cohen’s d = 3.22) and d′ (Cohen’s d = 3.22). The 50 final test items produced comparable results for accuracy, d′, and response latency. For both versions of the test, score scale reliability was high for both polyp and non-polyp items (α = .82 to .97).

Conclusions

The observed experienced–novice differences support the construct validity of the performance measures derived from the tests, indicating that polyp recognition skill can be quantified objectively. The final test may potentially be used to assess trainees, but test sensitivity may be insufficient to make fine-grained distinctions between different skill levels among experienced colonoscopists. More sensitive future tests may provide a valuable supplement to clinical detection rates, allowing objective comparisons between skilled colonoscopists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Kaminski MF, Regula J, Kraszewska E, Polkowski M, Wojciechowska U, Didkowska J, Zwierko M, Rupinski M, Nowacki MP, Butruk E (2010) Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 362(19):1795–1803

    Article  CAS  PubMed  Google Scholar 

  2. Corley DA, Jensen CD, Marks AR, Zhao WK, Lee JK, Doubeni CA, Zauber AG, de Boer J, Fireman BH, Schottinger JE, Quinn VP, Ghai NR, Levin TR, Quesenberry CP (2014) Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 370(14):1298–1306

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Hewett DG, Kahi CJ, Rex DK (2010) Does colonoscopy work? J Natl Compr Cancer Netw 8:67–76

    Article  Google Scholar 

  4. van Rijn JC, Reitsma JB, Stoker J, Bossuyt PM, van Deventer SJ, Dekker E (2006) Polyp miss rate determined by tandem colonoscopy: a systematic review. Am J Gastroenterol 101(2):343–350

    Article  PubMed  Google Scholar 

  5. Rex DK, Cutler CS, Lemmel GT, Rahmani EY, Clark DW, Helper DJ, Lehman GA, Mark DG (1997) Colonoscopic miss rates of adenomas determined by back-to-back colonoscopies. Gastroenterology 112(1):24–28

    Article  CAS  PubMed  Google Scholar 

  6. Chen SC, Rex DK (2008) Variable detection of nonadenomatous polyps by individual endoscopists at colonoscopy and correlation with adenoma detection. J Clin Gastroenterol 42(6):704–707

    Article  PubMed  Google Scholar 

  7. Hetzel JT, Huang CS, Coukos JA, Omstead K, Cerda SR, Yang S, O’Brien MJ, Farraye FA (2010) Variation in the detection of serrated polyps in an average risk colorectal cancer screening cohort. Am J Gastroenterol 105:2656–2664

    Article  PubMed  Google Scholar 

  8. Kahi CJ, Hewett DG, Norton DL, Eckert GJ, Rex DK (2010) Prevalence and variable detection of proximal colon serrated polyps during screening colonoscopy. Clin Gastroenterol Hepatol 9(1):42–46

    Article  PubMed  Google Scholar 

  9. Barclay RL, Vicari JJ, Doughty AS, Johanson JF, Greenlaw RL (2006) Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 355(24):2533–2541

    Article  CAS  PubMed  Google Scholar 

  10. Hewett DG, Kahi CJ, Rex DK (2010) Efficacy and effectiveness of colonoscopy: how do we bridge the gap? Gastrointest Endosc Clin N Am 20(4):673–684

    Article  PubMed  Google Scholar 

  11. Lin OS (2011) From good to great: further thoughts on improving quality during routine colonoscopy. Gastrointest Endosc 73(3):464–466

    Article  PubMed  Google Scholar 

  12. Rex DK (2000) Colonoscopic withdrawal technique is associated with adenoma miss rates. Gastrointest Endosc 51(1):33–36

    Article  CAS  PubMed  Google Scholar 

  13. Lee RH, Tang RS, Muthusamy VR, Ho SB, Shah NK, Wetzel L, Bain AS, Mackintosh EE, Paek AM, Crissien AM, Saraf LJ, Kalmaz DM, Savides TJ (2011) Quality of colonoscopy withdrawal technique and variability in adenoma detection rates (with videos). Gastrointest Endosc 74(1):128–134

    Article  PubMed  Google Scholar 

  14. Rex DK (2007) Who is the best colonoscopist? Gastrointest Endosc 65(1):145–150

    Article  PubMed  Google Scholar 

  15. Rex DK (2009) Narrow-band imaging without optical magnification for histologic analysis of colorectal polyps. Gastroenterology 136(4):1174–1181

    Article  PubMed  Google Scholar 

  16. Raghavendra M, Hewett DG, Rex DK (2010) Differentiating adenomas from hyperplastic colorectal polyps: narrow-band imaging can be learned in 20 minutes. Gastrointest Endosc 72(3):572–576

    Article  PubMed  Google Scholar 

  17. Ignjatovic A, East JE, Suzuki N, Vance M, Guenther T, Saunders BP (2009) Optical diagnosis of small colorectal polyps at routine colonoscopy (Detect InSpect ChAracterise Resect and Discard; DISCARD trial): a prospective cohort study. Lancet Oncol 10(12):1171–1178

    Article  PubMed  Google Scholar 

  18. Rastogi A, Rao DS, Gupta N, Grisolano SW, Buckles DC, Sidorenko E, Bonino J, Matsuda T, Dekker E, Kaltenbach T, Singh R, Wani S, Sharma P, Olyaee MS, Bansal A, East JE (2014) Impact of a computer-based teaching module on characterization of diminutive colon polyps by using narrow-band imaging by non-experts in academic and community practice: a video-based study. Gastrointest Endosc 79(3):390–398

    Article  PubMed  Google Scholar 

  19. Rex DK, Hewett DG, Snover DC (2010) Detection targets for colonoscopy: from variable detection to validation. Am J Gastroenterol 105(12):2665–2669

    Article  PubMed  Google Scholar 

  20. Rosty C, Hewett DG, Brown IS, Leggett BA, Whitehall VL (2013) Serrated polyps of the large intestine: current understanding of diagnosis, pathogenesis, and clinical management. J Gastroenterol 48(3):287–302

    Article  PubMed  Google Scholar 

  21. Nodine CF, Mello-Thoms C, Kundel HL, Weinstein SP (2002) Time course of perception and decision making during mammographic interpretation. AJR Am J Roentgenol 179(4):575–585

    Article  Google Scholar 

  22. McCarley JS, Kramer AF, Wickens CD, Vidoni ED, Boot WR (2004) Visual skills in airport-security screening. Psychol Sci 15(5):80–88

    Article  Google Scholar 

  23. Wickens CD, Carswell CM (2006) Information processing. In: Salvendy G (ed) Handbook of human factors and ergonomics, 3rd edn. Wiley, Hoboken, pp 111–149

    Chapter  Google Scholar 

  24. Sowden PT, Davies IRL, Roling P (2000) Perceptual learning of the detection of features in X-ray images: a functional role for improvements in adults’ visual sensitivity? J Exp Psychol Hum Percept Perform 26(1):379–390

    Article  CAS  PubMed  Google Scholar 

  25. Fairhurst K, Strickland A, Maddern GJ (2011) Simulation speak. J Surg Ed 68(5):382–386

    Article  Google Scholar 

  26. Plooy AM, Hill A, Horswill MS, Cresp AS, Watson MO, Ooi SY, Riek S, Wallis GM, Burgess-Limerick R, Hewett DG (2012) Construct validation of a physical model colonoscopy simulator. Gastrointest Endosc 76(1):144–150

    Article  PubMed  Google Scholar 

  27. Wetton MA, Hill A, Horswill MS (2011) The development and validation of a hazard perception test for use in driver licensing. Accid Anal Prev 43(5):1759–1770

    Article  PubMed  Google Scholar 

  28. Ignjatovic A, Thomas-Gibson S, East JE, Haycock A, Bassett P, Bhandari P, Man R, Suzuki N, Saunders BP (2011) Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps. Gastrointest Endosc 73(1):128–133

    Article  PubMed  Google Scholar 

  29. Cohen RJ, Swerdlik ME (1999) Psychological testing and assessment: an introduction to tests and measurement. Mayfield Publishing Company, Mountain View

    Google Scholar 

  30. Anastasi A, Urbina S (1997) Psychological testing, 7th edn. Prentice-Hall, Upper Saddle River

    Google Scholar 

  31. Hewett DG, Kaltenbach T, Sano Y, Tanaka S, Saunders BP, Ponchon T, Soetikno R, Rex DK (2012) Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging. Gastroenterology 143 (3):599–607 e591

  32. Hewett DG, Rex DK (2011) Miss rate of right-sided colon examination during colonoscopy defined by retroflexion: an observational study. Gastrointest Endosc 74(2):246–252

    Article  PubMed  Google Scholar 

  33. Hewett DG, Rex DK (2012) Inspection on instrument insertion during colonoscopy: a randomized controlled trial. Gastrointest Endosc 76(2):381–387

    Article  PubMed  Google Scholar 

  34. Hewett DG, Rex DK (2010) Cap-fitted colonoscopy: a randomized, tandem colonoscopy study of adenoma miss rates. Gastrointest Endosc 72(4):775–781

    Article  PubMed  Google Scholar 

  35. Hewett DG, Huffman ME, Rex DK (2012) Leaving distal colorectal hyperplastic polyps in place can be achieved with high accuracy by using narrow-band imaging: an observational study. Gastrointest Endosc 76(2):374–380

    Article  PubMed  Google Scholar 

  36. Wickens TD (2002) Elementary signal detection theory. Oxford University Press, Oxford

    Google Scholar 

  37. Houx PJ, Jolles J (1993) Age-related decline of psychomotor speed: effects of age, brain health, sex, and education. Percept Mot Skills 76(1):195–211

    Article  CAS  PubMed  Google Scholar 

  38. Poulsen AA, Horswill MS, Wetton MA, Hill A, Lim SM (2010) A brief office-based hazard perception intervention for drivers with ADHD symptoms. Aust N Z J Psychiatry 44(6):528–534

    PubMed  Google Scholar 

  39. Waye JD, Rex D, Williams CE (eds) (2009) Colonoscopy: principles and practice, 2nd edn. Wiley-Blackwell, Hoboken, NJ

    Google Scholar 

  40. Yamada T (ed) (2009) Atlas of gastroenterology. Wiley-Blackwell, Hoboken

    Google Scholar 

  41. Sharma M, Petosa RL (2012) Measurement and evaluation for health educators. Jones & Bartlett Learning, Burlington

    Google Scholar 

  42. DeVellis RF (1991) Scale development: theory and applications. Sage Publications, Newbury Park

    Google Scholar 

  43. George D, Mallery P (2003) SPSS for Windows step by step: a simple guide and reference. Allyn & Bacon, Boston

    Google Scholar 

  44. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. L. Erlbaum Associates, Hillsdale

    Google Scholar 

  45. Rosnow RL, Rosenthal R (1996) Computing contrasts, effect sizes, and counternulls on other people’s published data: general procedures for research consumers. Psychol Meth 1(4):331–340

    Article  Google Scholar 

  46. Cohen J (1992) A power primer. Psychol Bull 112(1):155–159

    Article  CAS  PubMed  Google Scholar 

  47. Church J (2008) Adenoma detection rate and the quality of colonoscopy: the sword has two edges. Dis Colon Rectum 51(5):520–523

    Article  PubMed  Google Scholar 

  48. Hewett DG, Rex DK (2010) Improving colonoscopy quality through health-care payment reform. Am J Gastroenterol 105(9):1925–1933

    Article  PubMed  Google Scholar 

  49. Sedlack RE (2010) The Mayo colonoscopy skills assessment tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 72(6):1125–1133

    Article  PubMed  Google Scholar 

  50. Lee SH, Chung IK, Kim SJ, Kim JO, Ko BM, Hwangbo Y, Kim WH, Park DH, Lee SK, Park CH, Baek IH, Park DI, Park SJ, Ji JS, Jang BI, Jeen YT, Shin JE, Byeon JS, Eun CS, Han DS (2008) An adequate level of training for technical competence in screening and diagnostic colonoscopy: a prospective multicenter evaluation of the learning curve. Gastrointest Endosc 67(4):683–689

    Article  PubMed  Google Scholar 

  51. Zupanc CM, Burgess-Limerick R, Hill A, Riek S, Wallis GM, Plooy AM, Horswill MS, Watson MO, Hewett DG (2015) A competency framework for colonoscopy training derived from cognitive task analysis techniques and expert review. BMC Med Educ 15:216

    Article  PubMed  PubMed Central  Google Scholar 

  52. Spring KJ, Zhao ZZ, Karamatic R, Walsh MD, Whitehall VL, Pike T, Simms LA, Young J, James M, Montgomery GW, Appleyard M, Hewett D, Togashi K, Jass JR, Leggett BA (2006) High prevalence of sessile serrated adenomas with BRAF mutations: a prospective study of patients undergoing colonoscopy. Gastroenterology 131(5):1400–1407

    Article  CAS  PubMed  Google Scholar 

  53. Leggett B, Whitehall V (2010) Role of the serrated pathway in colorectal cancer pathogenesis. Gastroenterology 138(6):2088–2100

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank Natasha Innes for assisting with data collection and Philippa Neary for reviewing a draft of this paper.

Funding

This research was supported by the Australian Government Department of Health and Ageing. David G. Hewett was supported by a Sylvia and Charles Viertel Charitable Foundation Clinical Investigatorship. Guy M. Wallis was supported by an Australian Research Council Future Fellowship (FT100100020). The funders had no role in study design, data collection and analysis, the decision to publish, or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Clinical Skills Development Service, Metro North Hospital and Health Service, Brisbane, Australia

    Andrew Hill, Marcus O. Watson & David G. Hewett

  2. School of Psychology, The University of Queensland, Brisbane, Australia

    Andrew Hill, Mark S. Horswill & Marcus O. Watson

  3. School of Human Movement Studies, The University of Queensland, Brisbane, Australia

    Annaliese M. Plooy, Guy M. Wallis, Stephan Riek & Robin Burgess-Limerick

  4. School of Medicine, The University of Queensland, Brisbane, Australia

    Marcus O. Watson, Lachlan N. Rowlands & David G. Hewett

Authors
  1. Andrew Hill
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark S. Horswill
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Annaliese M. Plooy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marcus O. Watson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lachlan N. Rowlands
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guy M. Wallis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stephan Riek
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Robin Burgess-Limerick
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. David G. Hewett
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew Hill.

Ethics declarations

Disclosures

Andrew Hill, Mark S. Horswill, Annaliese M. Plooy, Marcus O. Watson, Lachlan N. Rowlands, Guy M. Wallis, Stephan Riek, Robin Burgess-Limerick and David G. Hewett have no conflicts of interest or financial ties to disclose.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hill, A., Horswill, M.S., Plooy, A.M. et al. Assessment of colorectal polyp recognition skill: development and validation of an objective test. Surg Endosc 31, 2426–2436 (2017). https://doi.org/10.1007/s00464-016-5243-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-016-5243-9

Keywords

Search

Navigation