Skip to main content
SpringerLink
Log in

A novel endoscopy image fusion system: combine white light imaging and compound band imaging

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

White light imaging (WLI) is a commonly seen examination mode in endoscopy. The particular light in compound band imaging (CBI) can highlight delicate structures, such as capillaries and tiny structures on the mucosal surface. These two modes complement each other, and doctors switch between them manually to complete the examination. This paper proposes an endoscopy image fusion system to combine WLI and CBI.

Methods

We add a real-time rotatable color wheel in the light source device of the AQ-200 endoscopy system to achieve rapid imaging of two modes at the same position of living tissue. The two images corresponding to the pixel level can avoid registration and lay the foundation for image fusion. We propose a multi-scale image fusion framework, which involves Laplacian pyramid (LP) and convolutional sparse representation (CSR) and strengthens the details in the fusion rule.

Results

Volunteer experiments and ex vivo pig stomach trials are conducted to verify the feasibility of our proposed system. We also conduct comparative experiments with other image fusion methods, evaluate the quality of the fused images, and verify the effectiveness of our fusion framework. The results show that our fused image has rich details, high color contrast, apparent structures, and clear lesion boundaries.

Conclusion

An endoscopy image fusion system is proposed, which does not change the doctor's operation and makes the fusion of WLI and CBI optical staining technology a reality. We change the light source device of the endoscope, propose an image fusion framework, and verify the feasibility and effectiveness of our scheme. Our method fully integrates the advantages of WLI and CBI, which can help doctors make more accurate judgments than before. The endoscopy image fusion system is of great significance for improving the detection rate of early lesions and has broad application prospects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Nakamoto S, Sakai Y, Kasanuki J, Kondo F, Ooka Y, Kato K, Arai M, Suzuki T, Matsumura T, Bekku D (2009) Indications for the use of endoscopic mucosal resection for early gastric cancer in Japan: a comparative study with endoscopic submucosal dissection. Endoscopy 41(09):746–750

    Article  CAS  PubMed  Google Scholar 

  2. Gono K, Yamazaki K, Doguchi N, Nonami T, Obi T, Yamaguchi M, Ohyama N, Machida H, Sano Y, Yoshida S (2003) Endoscopic observation of tissue by narrowband illumination. Opt Rev 10(4):211–215

    Article  Google Scholar 

  3. Sturm MB, Wang TD (2015) Emerging optical methods for surveillance of Barrett’s oesophagus. Gut 64(11):1816–1823

    Article  CAS  PubMed  Google Scholar 

  4. Song LMWK, Adler DG, Chand B, Conway JD, Croffie JM, DiSario JA, Mishkin DS, Shah RJ, Somogyi L, Tierney WM (2007) Chromoendoscopy. Gastrointest Endosc 66(4):639–649

    Article  Google Scholar 

  5. Kurniawan N, Keuchel M (2017) Flexible gastro-intestinal endoscopy—clinical challenges and technical achievements. Comput Struct Biotechnol J 15:168–179

    Article  PubMed  PubMed Central  Google Scholar 

  6. Jang J-Y (2015) The past, present, and future of image-enhanced endoscopy. Clinical endoscopy 48(6):466–475

    Article  PubMed  PubMed Central  Google Scholar 

  7. Joren R, Oldenburg B (2015) Surveillance of long-standing colitis: the role of image-enhanced endoscopy. Best Pract Res Clin Gastroenterol 29(4):687–697

    Article  Google Scholar 

  8. Gono K, Obi T, Yamaguchi M, Oyama N, Machida H, Sano Y, Yoshida S, Hamamoto Y, Endo T (2004) Appearance of enhanced tissue features in narrow-band endoscopic imaging. J Biomed Opt 9(3):568–577

    Article  PubMed  Google Scholar 

  9. Yoshida N, Dohi O, Inoue K, Yasuda R, Murakami T, Hirose R, Inoue K, Naito Y, Inada Y, Ogiso K (2019) Blue laser imaging, blue light imaging, and linked color imaging for the detection and characterization of colorectal tumors. Gut and liver 13(2):140

    Article  PubMed  Google Scholar 

  10. Kodashima S, Fujishiro M (2010) Novel image-enhanced endoscopy with i-scan technology. World J Gastroenterol: WJG 16(9):1043

    Article  PubMed  PubMed Central  Google Scholar 

  11. Coriat R, Chryssostalis A, Zeitoun J, Deyra J, Gaudric M, Prat F, Chaussade S (2008) Computed virtual chromoendoscopy system (FICE): a new tool for upper endoscopy? Gastroenterol Clin Biol 32(4):363–369

    Article  CAS  PubMed  Google Scholar 

  12. Jacques SL (2013) Optical properties of biological tissues: a review. Phys Med Biol 58(11):R37

    Article  ADS  PubMed  Google Scholar 

  13. Tuchin VV (ed) (2015) Tissue optics2015: Society of Photo-Optical Instrumentation Engineers (SPIE) Bellingham, WA, USA.

  14. Prahl S (1999) Optical absorption of hemoglobin. Or Med Center News

  15. Kashida H (2019) Endoscopic diagnosis of sessile serrated polyp: a systematic review. Dig Endosc 31(1):16–23

    Article  PubMed  Google Scholar 

  16. He Z, Wang P, Liang Y, Fu Z, Ye X (2021) Clinically available optical imaging technologies in endoscopic lesion detection: current status and future perspective. J Healthcare Eng 2021:1–27

    CAS  Google Scholar 

  17. Kaur H, Koundal D, Kadyan V (2021) Image fusion techniques: a survey. Arch Comput Methods Eng 28(7):4425–4447

    Article  PubMed  PubMed Central  Google Scholar 

  18. Manviya M, Bharti J (2020) Image fusion survey: a comprehensive and detailed analysis of image fusion techniques. Social Networking and Computational Intelligence. Springer, Cham, pp 649–659.

  19. Song L, Lin Y, Feng W, Zhao M (eds) (2009) A novel automatic weighted image fusion algorithm. In: 2009 International workshop on intelligent systems and applications. IEEE, New York.

  20. Jasiunas MD, Kearney DA, Hopf J, Wigley GB (eds) (2002) Image fusion for uninhabited airborne vehicles. In: 2002 IEEE international conference on field-programmable technology, 2002 (FPT). Proceedings. IEEE, New York.

  21. Harris JR, Murray R, Hirose T (1990) IHS transform for the integration of radar imagery with other remotely sensed data. Photogramm Eng Remote Sens 56(12):1631–1641

    ADS  Google Scholar 

  22. Wang H-q, Xing H (eds) (2009) Multi-mode medical image fusion algorithm based on principal component analysis. In: 2009 International symposium on computer network and multimedia technology. IEEE, New York.

  23. Sahu A, Bhateja V, Krishn A (eds) (2014) Medical image fusion with Laplacian pyramids. In: 2014 International conference on medical imaging, m-health and emerging communication systems (MedCom). IEEE, New York

  24. Singh R, Khare A (2013) Multiscale medical image fusion in wavelet domain. Sci World J 2013

  25. Yang B, Li S (2009) Multifocus image fusion and restoration with sparse representation. IEEE Trans Instrum Meas 59(4):884–892

    Article  ADS  Google Scholar 

  26. Du J, Li W, Xiao B, Nawaz Q (2016) Union Laplacian pyramid with multiple features for medical image fusion. Neurocomputing 194:326–339

    Article  Google Scholar 

  27. Liu Y, Chen X, Ward RK, Wang ZJ (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886

    Article  ADS  Google Scholar 

  28. Wohlberg B (2015) Efficient algorithms for convolutional sparse representations. IEEE Trans Image Process 25(1):301–315

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  29. Olkkonen H, Pesola P (1996) Gaussian pyramid wavelet transform for multiresolution analysis of images. Graph Models Image Process 58(4):394–398

    Article  Google Scholar 

  30. Zhang Q, Liu Y, Blum RS, Han J, Tao D (2018) Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: a review. Inform Fusion 40:57–75

    Article  Google Scholar 

  31. Easley G, Labate D, Lim W-Q (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46

    Article  MathSciNet  Google Scholar 

  32. Rolland JP, Vo V, Bloss B, Abbey CK (2000) Fast algorithms for histogram matching: application to texture synthesis. J Electron Imaging 9(1):39–45

    Article  ADS  Google Scholar 

  33. Yoshida N, Dohi O, Inoue K, Yasuda R, Murakami T, Hirose R, Naito Y, Inada Y, Ogiso K, Morinaga Y (2019) Blue laser imaging, blue light imaging, and linked color imaging for the detection and characterization of colorectal tumors. Gut Liver 13(2):140

    Article  PubMed  Google Scholar 

  34. Fujii T, Ono A (2002) Is adaptive index of hemoglobin color enhancement effective in detecting small depressed type colorectal cancers? Dig Endosc 14:S58–S61

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge supports from National Key Research and Development Program of China (2022YFC2405200), National Natural Science Foundation of China (82027807, U22A2051), and Beijing Municipal Natural Science Foundation (7212202).

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, China

    Shipeng Zhang, Xinran Zhang & Hongen Liao

  2. Department of Biomedical Engineering, College of Future Technology, Peking University, Beijing, 100871, China

    Ye Fu & Tianyu Xie

  3. Beijing Tsinghua Changgung Hospital, Beijing, 102218, China

    Zhe Zhao

  4. School of Clinical Medicine, Tsinghua University, Beijing, 100084, China

    Zhe Zhao

Authors
  1. Shipeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ye Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinran Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianyu Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhe Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongen Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhe Zhao or Hongen Liao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

For this type of study, ethical approval is not required.

Informed consent

This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Figs. 11, 12, and Tables 6 and 7.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Fu, Y., Zhang, X. et al. A novel endoscopy image fusion system: combine white light imaging and compound band imaging. Int J CARS 19, 331–344 (2024). https://doi.org/10.1007/s11548-023-02988-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-02988-x

Keywords

Search

Navigation