Skip to main content

Advertisement

SpringerLink
Log in

An Augmented Reality Endoscope System for Ureter Position Detection

  • Image & Signal Processing
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Iatrogenic injury of ureter in the clinical operation may cause the serious complication and kidney damage. To avoid such a medical accident, it is necessary to provide the ureter position information to the doctor. For the detection of ureter position, an ureter position detection and display system with the augmented ris proposed to detect the ureter that is covered by human tissue. There are two key issues which should be considered in this new system. One is how to detect the covered ureter that cannot be captured by the electronic endoscope and the other is how to display the ureter position that provides stable and high-quality images. Simultaneously, any delayed processing of the system should disturb the surgery. The aided hardware detection method and target detection algorithms are proposed in this system. To mark the ureter position, a surface-lighting plastic optical fiber (POF) with the encoded light-emitting diode (LED) light is used to indicate the ureter position. The monochrome channel filtering algorithm (MCFA) is proposed to locate the ureter region more precisely. The ureter position is extracted using the proposed automatic region growing algorithm (ARGA) that utilizes the statistical information of the monochrome channel for the selection of growing seed point. In addition, according to the pulse signal of encoded light, the recognition of bright and dark frames based on the aided hardware (BDAH) is proposed to expedite the processing speed. Experimental results demonstrate that the proposed endoscope system can identify 92.04% ureter region in average.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ševo, I., Avramović, A., Balasingham, I., Elle, O., Bergsland, J., and Aabakken, L., Edge density based automatic detection of inflammation in colonoscopy videos. Comput. Biol. Med. 72:138–150, 2016.

    Article  PubMed  Google Scholar 

  2. Reynisson, P. J., Leira, H. O., Hernes, T. N., Hofstad, E. F., Scali, M., Sorger, H., Amundsen, T., Lindseth, F., and Langø, T., Navigated bronchoscopy: a technical review. Journal of Bronchology & Interventional Pulmonology 21(3):242–264, 2014.

    Article  Google Scholar 

  3. Lambert, R., Prevention of gastrointestinal cancer by surveillance endoscopy. EPMA J. 1(3):473–483, 2010.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Lurie, K. L., Smith, G. T., Khan, S. A., Liao, J. C., and Ellerbee, A. K., Three-dimensional, distendable bladder phantom for optical coherence tomography and white light cystoscopy. J. Biomed. Opt. 19(3):036 009-036 009, 2014.

    Article  Google Scholar 

  5. Selka, F., Nicolau, S., Agnus, V., Bessaid, A., Marescaux, J., and Soler, L., Context-specific selection of algorithms for recursive feature tracking in endoscopic image using a new methodology. Comput. Med. Imaging Graph. 40:49–61, 2015.

    Article  PubMed  CAS  Google Scholar 

  6. Oh, S. Y., Kwon, S., Lee, K. G., Suh, Y. S., Choe, H. N., Kong, S. H., Lee, H. J., Kim, W. H., and Yang, H. K., Outcomes of minimally invasive surgery for early gastric cancer are comparable with those for open surgery: analysis of 1,013 minimally invasive surgeries at a single institution. Surg. Endosc. 28(3):789–795, 2014.

    Article  PubMed  Google Scholar 

  7. Burdall, O. C., Boddy, A. P., Fullick, J., Blazeby, J., Krysztopik, R., Streets, C., Hollowood, A., Barham, C. P., and Titcomb, D., A comparative study of survival after minimally invasive and open oesophagectomy. Surg. Endosc. 29(2):431–437, 2015.

    Article  PubMed  Google Scholar 

  8. Clayman, R. V., Kavoussi, L. R., Soper, N. J., Dierks, S. M., Meretyk, S., Darcy, M. D., Roemer, F. D., Pingleton, E. D., Thomson, P. G., and Long, S. R., Laparoscopic nephrectomy: initial case report. J. Urol. 197(2):S182–S186, 2017.

    Article  PubMed  Google Scholar 

  9. Martin-Perez, B., Andrade-Ribeiro, G., Hunter, L., and Atallah, S., A systematic review of transanal minimally invasive surgery (tamis) from 2010 to 2013. Tech. Coloproctol. 18(9):775–788, 2014.

    Article  PubMed  CAS  Google Scholar 

  10. Van den Haak, L., Alleblas, C., Nieboer, T., Rhemrev, J., and Jansen, F., Efficacy and safety of uterine manipulators in laparoscopic surgery: a review. Arch. Gynecol. Obstet. 292(5):1003–1011, 2015.

    Article  Google Scholar 

  11. Aronson, M. P., and Bose, T. M., Urinary tract injury in pelvic surgery. Clin. Obstet. Gynecol. 45(2):428–438, 2002.

    Article  PubMed  Google Scholar 

  12. Harkki-Siren, P., Sjoberg, J., and Tiitinen, A., Urinary tract injuries after hysterectomy. Obstet. Gynecol. 92(1):113–118, 1998.

    Article  PubMed  CAS  Google Scholar 

  13. Parpala-Spårman, T., Paananen, I., Santala, M., Ohtonen, P., and Hellström, P., Increasing numbers of ureteric injuries after the introduction of laparoscopic surgery. Scand. J. Urol. Nephrol. 42(5):422–427, 2008.

    Article  PubMed  Google Scholar 

  14. Orr, W. S., Pisters, L. L., and Rodriguez-Bigas, M.A.: Intraoperative ureteral injury 34. Gastrointestinal Surgery: Management of Complex Perioperative Complications, p 361, 2015

  15. Acher, C., and Agarwal, S.: Injury of the kidney, ureter, and bladder. In: Penetrating trauma, Springer, pp. 387–396, 2017.

  16. Wu, C. J., Tseng, C. W., and Wu, M. P., Laparoscopic subtotal hysterectomy in the era of minimally invasive surgery. Gynecology and Minimally Invasive Therapy 4(1):8–13, 2015.

    Article  Google Scholar 

  17. Janssen, P. F., Brölmann, H. A., and Huirne, J. A., Causes and prevention of laparoscopic ureter injuries: an analysis of 31 cases during laparoscopic hysterectomy in the netherlands. Surg. Endosc. 27(3):946–956, 2013.

    Article  PubMed  Google Scholar 

  18. Cuesta, M. A.: Case on ureter lesion during laparoscopic low anterior resection. In: Case Studies of Postoperative Complications after Digestive Surgery, Springer, pp. 407–413, 2014.

  19. Schimpf, M., and Gottenger, E., Universal ureteral stent placement at hysterectomy to identify ureteral injury: a decision analysis. J. Wagner, BJOG: An International Journal of Obstetrics & Gynaecology 115(9):1151–1158, 2008.

    Article  CAS  Google Scholar 

  20. Fu, W. J., Wang, Z. X., Li, G., Cui, F. Z., Zhang, Y., and Zhang, X., Comparison of a biodegradable ureteral stent versus the traditional double-j stent for the treatment of ureteral injury: an experimental study. Biomed. Mater. 7(6):065002, 2012.

    Article  PubMed  CAS  Google Scholar 

  21. Chahin, F., Dwivedi, A. J., Paramesh, A., Chau, W., Agrawal, S., Chahin, C., Kumar, A., Tootla, A., Tootla, F., and Silva, Y. J., The implications of lighted ureteral stenting in laparoscopic colectomy. Jsls 6(1):49–52, 2002.

    PubMed  PubMed Central  Google Scholar 

  22. Kwon, I. G., Cho, I., Guner, A., Choi, Y. Y., Shin, H. B., Kim, H. I., An, J. Y., Cheong, J. H., Noh, S. H., and Hyung, W. J., Minimally invasive surgery for remnant gastric cancer: a comparison with open surgery. Surg. Endosc. 28(8):2452–2458, 2014.

    Article  PubMed  Google Scholar 

  23. Uttley, L., Campbell, F., Rhodes, M., Cantrell, A., Stegenga, H., and Lloyd-Jones, M., Minimally invasive oesophagectomy versus open surgery: is there an advantage? Surg. Endosc. 27(3):724–731, 2013.

    Article  PubMed  Google Scholar 

  24. Tanaka, E., Ohnishi, S., Laurence, R. G., Choi, H. S., Humblet, V., and Frangioni, J. V., Real-time intraoperative ureteral guidance using invisible near-infrared fluorescence. J. Urol. 178(5):2197–2202, 2007.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  25. Verbeek, F. P. R., Vorst, J. R. V. D., Schaafsma, B. E., Swijnenburg, R. J., Gaarenstroom, K. N., Elzevier, H. W., Velde, C. J. H. V. D., Frangioni, J. V., and Vahrmeijer, A. L., Intraoperative near infrared fluorescence guided identification of the ureters using low dose methylene blue: A first in human experience. J. Urol. 190(2):574–579, 2013.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  26. Al-Taher, M., van den Bos, J., Schols, R. M., Bouvy, N. D., and Stassen, L. P., Fluorescence ureteral visualization in human laparoscopic colorectal surgery using methylene blue. J. Laparoendosc. Adv. Surg. Tech. 26(11):870–875, 2016.

    Article  Google Scholar 

  27. Song, E., Yu, F., Liu, H., Cheng, N., Li, Y., Jin, L., and Hung, C. C., A novel endoscope system for position detection and depth estimation of the ureter. J. Med. Syst. 40(12):266, 2016.

    Article  PubMed  Google Scholar 

  28. Loukas, C., Lahanas, V., and Georgiou, E., An integrated approach to endoscopic instrument tracking for augmented reality applications in surgical simulation training. Int. J. Med. Rob. Comput. Assisted Surg 9(4):e34–e51, 2013.

    Article  Google Scholar 

  29. Yamamoto, T., Abolhassani, N., Jung, S., Okamura, A. M., and Judkins, T. N., Augmented reality and haptic interfaces for robot-assisted surgery. Int. J. Med. Rob. Comput. Assisted Surg. 8(1):45–56, 2012.

    Article  Google Scholar 

  30. Pagador, J. B., Sánchez, L., Sánchez, J., Bustos, P., Moreno, J., and Sánchez-margallo, F. M., Augmented reality haptic (arh): an approach of electromagnetic tracking in minimally invasive surgery. Int. J. Comput. Assist. Radiol. Surg. 6(2):257–263 , 2011.

    Article  PubMed  CAS  Google Scholar 

  31. Bernhardt, S., Nicolau, S. A., Agnus, V., Soler, L., Doignon, C., and Marescaux, J., Automatic localization of endoscope in intraoperative ct image: a simple approach to augmented reality guidance in laparoscopic surgery. Med. Image Anal. 30:130–143, 2016.

    Article  PubMed  Google Scholar 

  32. Katić, D., Wekerle, A. L., Görtler, J., Spengler, P., Bodenstedt, S., Röhl, S., Suwelack, S., Kenngott, H. G., Wagner, M., Müller-Stich, B. P., et al., Context-aware augmented reality in laparoscopic surgery. Comput. Med. Imaging Graph. 37(2):174–182, 2013.

    Article  PubMed  Google Scholar 

  33. Puerto-Souza, G. A., Cadeddu, J. A., and Mariottini, G. L., Toward long-term and accurate augmented-reality for monocular endoscopic videos. IEEE Trans. Biomed. Eng. 61(10):2609–2620, 2014.

    Article  PubMed  Google Scholar 

  34. Russ, J. C., Matey, J. R., Mallinckrodt, A. J., and Mckay, S., The image processing handbook. Comput. Phys. 8(2):177, 2002.

    Article  Google Scholar 

  35. Good, M. M., Abele, T. A., Balgobin, S., Montoya, T. I., McIntire, D., and Corton, M. M., Vascular and ureteral anatomy relative to the midsacral promontory. Am. J. Obstet. Gynecol. 208(6):486–e1, 2013.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by National Key R & D Program of China, No. 2017YFC0112804, National Natural Science Foundation of China under grant project No.61370179, the Fundamental Research Funds for the Central Universities, HUST: 2016YXZD018 and HUST: 2017JYCX038, Medical Clinical Science and Technology Development Fund of Jiangsu University, No. JLY20140051C, and Clinical Medicine Science and Technology Projects in Jiangsu province, No. BL2014056.

Author information

Authors and Affiliations

  1. School of Computer and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Feng Yu, Enmin Song, Hong Liu & Jun Zhu

  2. Department of Urology, Affiliated Kunsan Hospital of Jiangsu University, Kunshan, Jiangsu, 215300, China

    Yunlong Li

  3. Center for Machine Vision and Security Research, Kennesaw State University, Kennesaw, GA, USA

    Chih-Cheng Hung

Authors
  1. Feng Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enmin Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunlong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chih-Cheng Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hong Liu.

Ethics declarations

Conflict of interests

The authors declare no conflict of interest.

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

This article is part of the Topical Collection on Advanced Computational Intelligence and Soft Computing in Medical Imaging

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, F., Song, E., Liu, H. et al. An Augmented Reality Endoscope System for Ureter Position Detection. J Med Syst 42, 138 (2018). https://doi.org/10.1007/s10916-018-0992-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-018-0992-8

Keywords

Search

Navigation