Skip to main content
SpringerLink
Log in

Automated visibility map of the internal colon surface from colonoscopy video

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Optical colonoscopy is a prominent procedure by which clinicians examine the surface of the colon for cancerous polyps using a flexible colonoscope. One of the main concerns regarding the quality of the colonoscopy is to ensure that the whole colonic surface has been inspected for abnormalities. In this paper, we aim at estimating areas that have not been covered thoroughly by providing a map from the internal colon surface.

Methods

Camera parameters were estimated using optical flow between consecutive colonoscopy frames. A cylinder model was fitted to the colon structure using 3D pseudo stereo vision and projected into each frame. A circumferential band from the cylinder was extracted to unroll the internal colon surface (band image). By registering these band images, drift in estimating camera motion could be reduced, and a visibility map of the colon surface could be generated, revealing uncovered areas by the colonoscope. Hidden areas behind haustral folds were ignored in this study. The method was validated on simulated and actual colonoscopy videos. The realistic simulated videos were generated using a colonoscopy simulator with known ground truth, and the actual colonoscopy videos were manually assessed by a clinical expert.

Results

The proposed method obtained a sensitivity and precision of 98 and 96 % for detecting the number of uncovered areas on simulated data, whereas validation on real videos showed a sensitivity and precision of 96 and 78 %, respectively. Error in camera motion drift could be reduced by almost 50 % using results from band image registration.

Conclusion

Using a simple cylindrical model for the colon and reducing drift by registering band images allows for the generation of visibility maps. The current results also suggest that the provided feedback through the visibility map could enhance clinicians’ awareness of uncovered areas, which in return could reduce the probability of missing polyps.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Australian Institute of Health and Welfare. http://www.aihw.gov.au/. Accessed 16 Oct 2013

  2. World Health Organization (WHO) Fact sheet # 297: Cancer. In: WHO. http://www.who.int/mediacentre/factsheets/fs297/en/. Accessed 21 Oct 2015

  3. de Groen PC (2010) Advanced systems to assess colonoscopy. Gastrointest Endosc Clin N Am 20:699–716. doi:10.1016/j.giec.2010.07.012

    Article  PubMed  Google Scholar 

  4. Edakkanambeth Varayil J, Enders F, Tavanapong W, Oh J, Wong J, de Groen PC (2011) Colonoscopy: what endoscopists inspect under optimal conditions. Gastroenterology 140:S-718. doi:10.1016/S0016-5085(11)62982-X

  5. Cotton PB, Williams CB (eds) (2008) Colonoscopy and flexible sigmoidoscopy. In: Practical gastrointestinal endoscopy: the fundamentals, 5th edn. Blackwell Publishing Ltd, Oxford, pp 81–175. doi:10.1002/9780470987032.ch6

  6. Zauber AG, Winawer SJ, O’Brien MJ, Lansdorp-Vogelaar I, van Ballegooijen M, Hankey BF, Shi W, Bond JH, Schapiro M, Panish JF, Stewart ET, Waye JD (2012) Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med 366:687–696. doi:10.1056/NEJMoa1100370

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Barclay RL, Vicari JJ, Doughty AS, Johanson JF, Greenlaw RL (2006) Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. N Engl J Med 355:2533–2541. doi:10.1056/NEJMoa055498

    Article  CAS  PubMed  Google Scholar 

  8. Oh JH, Hwang S, Cao Y, Tavanapong W, Liu D, Wong J, de Groen PC (2009) Measuring objective quality of colonoscopy. IEEE Trans Biomed Eng 56:2190–2196. doi:10.1109/TBME.2008.2006035

    Article  PubMed  Google Scholar 

  9. Filip D (2012) Colometer: a real-time quality feedback system for screening colonoscopy. World J Gastroenterol 18:4270. doi:10.3748/wjg.v18.i32.4270

    Article  PubMed  PubMed Central  Google Scholar 

  10. Hong D, Tavanapong W, Wong J, Oh J, de Groen PC (2013) 3D reconstruction of virtual colon structures from colonoscopy images. Comput Med Imaging Graph. doi:10.1016/j.compmedimag.2013.10.005

    PubMed  Google Scholar 

  11. Bergen T, Wittenberg T (2015) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform 20:304–321. doi:10.1109/JBHI.2014.2384134

    Article  Google Scholar 

  12. Armin MA, De Visser H, Chetty G, Dumas C, Conlan D, Grimpen F, Salvado O (2015) Visibility map: a new method in evaluation quality of optical colonoscopy. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015. Springer, Cham, pp 396–404

    Chapter  Google Scholar 

  13. Kaufman A, Wang J (2008) 3D surface reconstruction from endoscopic videos. In: Linsen L, Hagen H, Hamann B (eds) Visualization in medicine and life sciences. Springer, Berlin, pp 61–74

    Chapter  Google Scholar 

  14. Koppel D, Chen C-I,Wang Y-F, Lee H, Gu J, Poirson A,Wolters R (2007) Toward automated model building from video in computer-assisted diagnoses in colonoscopy. In: Cleary KR, Miga MI (eds) Medical imaging 2007: visualization and image-guided procedures, vol 6509. San Diego, CA. doi:10.1117/12.709595

  15. Bao G, Pahlavan K, Mi L (2015) Hybrid localization of microrobotic endoscopic capsule inside small intestine by data fusion of vision and RF sensors. IEEE Sens J 15:2669–2678. doi:10.1109/JSEN.2014.2367495

    Article  Google Scholar 

  16. Drozdzal M, Seguí S, Vitrià J, Malagelada C, Azpiroz F, Radeva P (2013) Adaptable image cuts for motility inspection using WCE. Comput Med Imaging Graph 37:72–80. doi:10.1016/j.compmedimag.2012.09.002

    Article  PubMed  Google Scholar 

  17. Bao G, Mi L, Pahlavan K (2013) A video aided RF localization technique for the wireless capsule endoscope (WCE) inside small intestine. ACM, Brussels

    Book  Google Scholar 

  18. Mountney P, Stoyanov D, Davison A, Yang G-Z (2006) Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. In: Larsen R, Nielsen M, Sporring J (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2006. Springer, Berlin, pp 347–354

    Chapter  Google Scholar 

  19. Grasa OG, Civera J, Montiel JMM (2011) EKF monocular SLAM with relocalization for laparoscopic sequences. IEEE, Shanghai

    Book  Google Scholar 

  20. Puerto-Souza GA, Staranowicz AN, Bell CS, Valdastri P, Mariottini G-L (2014) A comparative study of ego-motion estimation algorithms for teleoperated robotic endoscopes. In: Luo X, Reichl T, Mirota D, Soper T (eds) Computer-assisted and robotic endoscopy. Springer, Cham, pp 64–76

    Google Scholar 

  21. Liu J, Subramanian KR, Yoo TS (2013) A robust method to track colonoscopy videos with non-informative images. Int J Comput Assist Radiol Surg 8:575–592. doi:10.1007/s11548-013-0814-x

    Article  PubMed  Google Scholar 

  22. Mori K, Deguchi D, Sugiyama J, Suenaga Y, Toriwaki J, Maurer CR, Takabatake H, Natori H (2002) Tracking of a bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images. A preliminary version of this paper was presented at the Medical Image Computing and Computer-Assisted Intervention (MICCAI) conference, Utrecht, The Netherlands (2001). Med Image Anal 6:321–336. doi:10.1016/S1361-8415(02)00089-0

  23. Rai L, Helferty JP, Higgins WE (2008) Combined video tracking and image-video registration for continuous bronchoscopic guidance. Int J Comput Assist Radiol Surg 3:315–329. doi:10.1007/s11548-008-0241-6

    Article  Google Scholar 

  24. Luó X, Feuerstein M, Deguchi D, Kitasaka T, Takabatake H, Mori K (2012) Development and comparison of new hybrid motion tracking for bronchoscopic navigation. Med Image Anal 16:577–596. doi:10.1016/j.media.2010.11.001

    Article  PubMed  Google Scholar 

  25. Valdastri P, Ciuti G, Verbeni A, Menciassi A, Dario P, Arezzo A, Morino M (2012) Magnetic air capsule robotic system: proof of concept of a novel approach for painless colonoscopy. Surg Endosc 26:1238–1246. doi:10.1007/s00464-011-2054-x

    Article  CAS  PubMed  Google Scholar 

  26. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge

    Google Scholar 

  27. Wang H, Mirota D, Ishii M, Hager GD (2008) Robust motion estimation and structure recovery from endoscopic image sequences with an Adaptive Scale Kernel Consensus estimator. IEEE, pp 1–7

  28. Scaramuzza D, Martinelli A, Siegwart R (2006) A flexible technique for accurate omnidirectional camera calibration and structure from motion. IEEE, p 45

  29. Shi J, Tomasi C (1994) Good features to track. In: IEEE computer vision and pattern recognition, Society Press, pp 593–600

  30. Armin MA, Chetty G, Jurgen F, Visser HD, Dumas C, Fazlollahi A, Grimpen F, Salvado O (2015) Uninformative frame detection in colonoscopy through motion, edge and color features. In: International workshop on computer-assisted and robotic. doi:10.1007/978-3-319-29965-5_15

  31. Torr PHS, Zisserman A (2000) MLESAC: a new robust estimator with application to estimating image geometry. Comput Vis Image Underst 78:138–156. doi:10.1006/cviu.1999.0832

    Article  Google Scholar 

  32. Nister D (2004) An efficient solution to the five-point relative pose problem. IEEE Trans Pattern Anal Mach Intell 26:756–770. doi:10.1109/TPAMI.2004.17

    Article  PubMed  Google Scholar 

  33. More J (1978) The Levenberg–Marquardt algorithm: implementation and theory. Springer, Berlin

    Google Scholar 

  34. Scaramuzza D, Fraundorfer F (2011) Visual odometry: Part I—The first 30 years and fundamentals. IEEE Robot Autom Mag 18:80–92. doi:10.1109/MRA.2011.943233

    Article  Google Scholar 

  35. Welch G, Bishop G (1995) An introduction to the Kalman filter. Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill

  36. Nagao J, Mori K, Enjouji T, Deguchi D, Kitasaka T, Suenaga Y, Hasegawa J, Toriwaki J, Takabatake H, Natori H (2004) Fast and accurate bronchoscope tracking using image registration and motion prediction. In: Barillot C, Haynor DR, Hellier P (eds) Medical Image Computing and Computer-Assisted Intervention—MICCAI 2004. Springer, Berlin, pp 551–558

    Chapter  Google Scholar 

  37. Rabbani T, Dijkman S, van den Heuvel F, Vosselman G (2007) An integrated approach for modelling and global registration of point clouds. ISPRS J Photogramm Remote Sens 61:355–370. doi:10.1016/j.isprsjprs.2006.09.006

    Article  Google Scholar 

  38. Zhen Z, Jinwu Q, Yanan Z, Linyong S (2006) An intelligent endoscopic navigation system. IEEE, pp 1653–1657

  39. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359. doi:10.1016/j.cviu.2007.09.014

    Article  Google Scholar 

  40. De Visser H, Passenger J, Conlan D, Russ C, Hellier D, Cheng M, Acosta O, Ourselin S, Salvado O (2010) Developing a next generation colonoscopy simulator. Int J Image Graph 10:203–217. doi:10.1142/S0219467810003731

    Article  Google Scholar 

  41. OpenGL—the industry standard for high performance graphics. https://www.opengl.org/. Accessed 1 Feb 2016

  42. Han J (2006) Data mining: concepts and techniques, 2nd edn. Elsevier; Morgan Kaufmann, Amsterdam; Boston

  43. Konda V, Chauhan SS, Abu Dayyeh BK, Hwang JH, Komanduri S, Manfredi MA, Maple JT, Murad FM, Siddiqui UD, Banerjee S (2015) Endoscopes and devices to improve colon polyp detection. Gastrointest Endosc 81:1122–1129. doi:10.1016/j.gie.2014.10.006

    Article  PubMed  Google Scholar 

  44. Zhou Jin, Das A, Li Feng, Li Baoxin (2008) Circular generalized cylinder fitting for 3D reconstruction in endoscopic imaging based on MRF. In: IEEE computer vision and pattern recognition workshop, pp 1–8

Download references

Author information

Authors and Affiliations

  1. HCT, University of Canberra, Bruce, Canberra, ACT, Australia

    Mohammad Ali Armin & Girija Chetty

  2. CSIRO Biomedical Informatics, The Australian e-Health Research Centre, Level 5, UQ Health Sciences, Brisbane, QLD, 4029, Australia

    Mohammad Ali Armin, Hans De Visser, Cedric Dumas & Olivier Salvado

  3. Department of Gastroenterology and Hepatology, Royal Brisbane and Women’s Hospital, Herston, Brisbane, QLD, Australia

    Florian Grimpen

Authors
  1. Mohammad Ali Armin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Girija Chetty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hans De Visser
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cedric Dumas
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Florian Grimpen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Olivier Salvado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Mohammad Ali Armin or Olivier Salvado.

Ethics declarations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required.

Conflict of interest

Aspects of the technology might be covered by a patent under review where some of the authors are inventors; the patent would be owned by their institution.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Armin, M.A., Chetty, G., De Visser, H. et al. Automated visibility map of the internal colon surface from colonoscopy video. Int J CARS 11, 1599–1610 (2016). https://doi.org/10.1007/s11548-016-1462-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-016-1462-8

Keywords

Search

Navigation