Skip to main content

Advertisement

SpringerLink
Log in

Registration of IRT and visible light images in neurosurgery: analysis and comparison of automatic intensity-based registration approaches

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The purpose of this study is to analyze and compare six automatic intensity-based registration methods for intraoperative infrared thermography (IRT) and visible light imaging (VIS/RGB). The practical requirement is to get a good performance of Euclidean distance between manually set landmarks in reference and target images as well as to achieve a high structural similarity index metric (SSIM) and peak signal-to-noise ratio (PSNR) with respect to the reference image.

Methods

In this study, preprocessing is applied to bring both image types to a similar intensity. Similarity transformation is employed to align roughly IRT and visible light images. Two optimizers and two measures are used in this process. Thereafter, due to locally different displacement of the brain surface through respiration and heartbeat, two non-rigid transformations are applied, and finally, a bicubic interpolation is carried out to compensate for the resulting estimated transformation. Performance was assessed using eleven image datasets. The registration accuracy of the different computational approaches was assessed based on SSIM and PSNR. Additionally, five concise landmarks for each dataset were selected manually in reference and target images and the Euclidean distance between the corresponding landmarks was compared.

Results

The results are showing that the combination of normalized intensity, mutual information measure with one-plus-one evolutionary optimizer in combination with Demon registration results in improved accuracy and performance as compared to all other methods tested here. Furthermore, the obtained results led to \(9.29\%\), \(11.59\%\), \(5.11\%\), \(4.62\%\), and \(12.92\%\) registrations for datasets 1, 2, 5, 7, and 8 with respect to the second best result by calculating the mean Euclidean distance of five landmarks.

Conclusions

We conclude that the mutual information measure with one-plus-one evolutionary optimizer in combination with Demon registration can achieve better accuracy and performance to those other methods mentioned here for automatic registration of IRT and visible light images in neurosurgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Albayrak B, Samdani AF, Black PM (2004) Intra-operative magnetic resonance imaging in neurosurgery. Acta Neurochir 146(6):543–557

    Article  CAS  PubMed  Google Scholar 

  2. Po-Hao Huang Abel, Jui-Chang Tsai, Lu-Ting Kuo, Chung-Wei Lee, Hong-Shiee Lai, Li-Kai Tsai, Sheng-Jean Huang, Chien-Min Chen, Yuan-Shen Chen, Hao-Yu Chuang (2014) Clinical application of perfusion computed tomography in neurosurgery. J Neurosurg 120(2):473–488

    Article  Google Scholar 

  3. Solís ST, de Quintana Schmidt C, Sáinchez JG, Portales IF, de Pedro MD, Berrocal VR, Valle RD, de trabajo de la SENEC G (2020) Intraoperative imaging in the neurosurgery operating theatre: a review of the most commonly used techniques for brain tumour surgery. Neurocirugía (English Edition)

  4. Moshaei-Nezhad Y, Müller J, Schnabel C, Kirsch M, Tetzlaff R (2021) A robust optical flow motion estimation and correction method for irt imaging in brain surgery. Quant Inf Thermo J 18(4):226–251

  5. Müller Jan, Müller Jens, Chen Fang, Tetzlaff Ronald, Müller Juliane, Böhl Elisa, Kirsch Matthias, Schnabel Christian (2018) Registration and fusion of thermographic and visual-light images in neurosurgery. IEEE Trans Biomed Circuits Syst 12(6):1313–1321

    Article  PubMed  Google Scholar 

  6. Hollmach J, Hoffmann N, Schnabel C, Küchler S, Sobottka S, Kirsch M, Schackert G, Koch E, Steiner G ( 2013) Highly sensitive time-resolved thermography and multivariate image analysis of the cerebral cortex for intrasurgical diagnostics. In: Photonic therapeutics and diagnostics IX, volume 8565, page 856550. International society for optics and photonics

  7. Gerald Steiner, Sobottka Stephan B, Edmund Koch, Gabriele Schackert, Matthias Kirsch (2011) Intraoperative imaging of cortical cerebral perfusion by time-resolved thermography and multivariate data analysis. J Biomed Opt 16(1):016001

    Article  Google Scholar 

  8. Sadeghi-Goughari Moslem, Mojra Afsaneh (2015) Intraoperative thermal imaging of brain tumors using a haptic-thermal robot with application in minimally invasive neurosurgery. Appl Therm Eng 91:600–610

    Article  Google Scholar 

  9. de Font-Réaulx E, López LR, Díaz LLG (2020) Infrared thermography mapping plus neuronavigation target location in an eloquent area cavernoma resection. Surgical neurology international, 11

  10. Lahiri BB, Bagavathiappan S, Jayakumar T, John Philip (2012) Medical applications of infrared thermography: a review. Inf Phys Technol 55(4):221–235

    Article  CAS  Google Scholar 

  11. Maurya L, Mahapatra P, Chawla D, Verma S (2020) An automatic thermal and visible image registration using a calibration rig. Recent trends in image and signal processing in computer vision, page 67

  12. Zitova Barbara, Flusser Jan (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000

    Article  Google Scholar 

  13. Harris CG, Stephens M (1988) A combined corner and edge detector. In: Alvey vision conference, vol 15, pp 10–5244. Citeseer

  14. Bay H, Tuytelaars T, Gool LV ( 2006) Surf: Speeded up robust features. In: European conference on computer vision, pp 404–417. Springer

  15. Rosten E, Drummond T (2005) Fusing points and lines for high performance tracking. In: Tenth IEEE international conference on computer vision (ICCV’05). Vol. 1, 2, pp. 1508–1515. Ieee

  16. Rublee E, Rabaud V, Konolige K, Bradski G (2011) Orb: an efficient alternative to sift or surf. In: 2011 international conference on computer vision, pp 2564–2571. Ieee

  17. Bom IMJ Van der, Klein S, Staring M, Homan R, Bartels LW, Pluim JPW (2011) Evaluation of optimization methods for intensity-based 2d-3d registration in x-ray guided interventions. In: Medical imaging 2011: image processing, vol 7962, p 796223. International Society for Optics and Photonics

  18. Penney Graeme P, Jürgen Weese, Little John A, Paul Desmedt, Hill Derek LG, Hawkes David J (1998) A comparison of similarity measures for use in 2-d-3-d medical image registration. IEEE Trans Med Imaging 17(4):586–595

    Article  Google Scholar 

  19. Penney Graeme P, Batchelor Philipp G, Hill Derek LG, Hawkes David J, Juergen Weese (2001) Validation of a two-to three-dimensional registration algorithm for aligning preoperative ct images and intraoperative fluoroscopy images. Med Phys 28(6):1024–1032

    Article  Google Scholar 

  20. Rahunathan S, Stredney D, Schmalbrock P, Clymer BD (2005) Image registration using rigid registration and maximization of mutual information. In: 13th annual medicine meets virtual reality conference

  21. Khamene Ali, Bloch Peter, Wein Wolfgang, Svatos Michelle, Sauer Frank (2006) Automatic registration of portal images and volumetric ct for patient positioning in radiation therapy. Med Image Anal 10(1):96–112

    Article  PubMed  Google Scholar 

  22. Hipwell John H, Penney Graeme P, McLaughlin Robert A, Kawal Rhode, Paul Summers, Cox Tim C, Byrne James V, Alison Noble J, Hawkes David J (2003) Intensity-based 2-d-3-d registration of cerebral angiograms. IEEE Trans Med Imaging 22(11):1417–1426

    Article  CAS  PubMed  Google Scholar 

  23. Oliveira Francisco PM, Joao Manuel RS, Tavares (2014) Medical image registration: a review. Comput Methods Biomech Biomed Eng 17(2):73–93

    Article  CAS  Google Scholar 

  24. Hill Derek LG, Batchelor Philipp G, Mark Holden, Hawkes David J (2001) Medical image registration. Phys Med Biol 46(3):R1

    Article  Google Scholar 

  25. Sotiras Aristeidis, Davatzikos Christos, Paragios Nikos (2013) Deformable medical image registration: a survey. IEEE Trans Med Imaging 32(7):1153–1190

    Article  PubMed  PubMed Central  Google Scholar 

  26. Ma Jiayi, Ma Yong, Li Chang (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178

    Article  Google Scholar 

  27. Öfverstedt Johan, Lindblad Joakim, Sladoje Nataša (2019) Fast and robust symmetric image registration based on distances combining intensity and spatial information. IEEE Trans Image Process 28(7):3584–3597

    Article  Google Scholar 

  28. Moshaei-Nezhad Y, Müller J, Tetzlaff R, Hoffmann N (2018) A new approach for motion estimation and correction of thermographic images in brain surgery. In: CNNA 2018; The 16th international workshop on cellular nanoscale networks and their applications, pp 1–4. VDE

  29. Moshaei-Nezhad Y, Müller J, Schnabel C, Kirsch M, Tetzlaff R (2021) Motion correction for irt imaging in neurosurgery: analysis and comparison of frequency-/filter-and intensity-based approaches. Inf Phys Technol 117:103804

  30. Tustison Nicholas J, Avants Brian B, Gee James C (2009) Directly manipulated free-form deformation image registration. IEEE Trans Image Process 18(3):624–635

  31. Stefan Klein, Marius Staring, Pluim Josien PW (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and b-splines. IEEE Trans Image Process 16(12):2879–2890

    Article  Google Scholar 

  32. Shackleford J, Kandasamy N, Sharp G (2013) High performance deformable image registration algorithms for manycore processors. Newnes

  33. Martin Styner, Christian Brechbuhler, Szckely G, Guido Gerig (2000) Parametric estimate of intensity inhomogeneities applied to mri. IEEE Trans Med Imag 19(3):153–165

    Article  Google Scholar 

  34. Nocedal J, Wright S (2006) Numerical optimization. Springer, Berlin

    Google Scholar 

  35. Reichenbach Stephen E, Frank Geng (2003) Two-dimensional cubic convolution. IEEE Trans Image Process 12(8):857–865

    Article  CAS  PubMed  Google Scholar 

  36. McClelland Jamie R, Hawkes David J, Tobias Schaeffter, King Andrew P (2013) Respiratory motion models: a review. Med Image Anal 17(1):19–42

    Article  Google Scholar 

  37. Zuiderveld K (1994) Contrast limited adaptive histogram equalization. Graphics gems, pp 474–485

  38. Ridler TW, Calvard S (1978) Picture thresholding using an iterative selection method. IEEE Trans Syst Man Cybern 8(8):630–632

  39. Russakoff DB, Tomasi C, Rohlfing T, Maurer CR (2004) Image similarity using mutual information of regions. In: European conference on computer vision, pp 596–607. Springer

  40. Thirion J-P (1998) Image matching as a diffusion process: an analogy with maxwell’s demons. Med Image Anal 2(3):243–260

  41. Vercauteren Tom, Pennec Xavier, Perchant Aymeric, Ayache Nicholas (2009) Diffeomorphic demons: efficient non-parametric image registration. Neuroimage 45(1):S61–S72

    Article  PubMed  Google Scholar 

  42. Daniel Rueckert, Sonoda Luke I, Carmel Hayes, Hill Derek LG, Leach Martin O, Hawkes David J (1999) Nonrigid registration using free-form deformations: application to breast mr images. IEEE Trans Med Imag 18(8):712–721

  43. Kabus Sven, Netsch Thomas, Fischer Bernd, Modersitzki Jan (2004) B-spline registration of 3d images with levenberg-marquardt optimization. In: Medical imaging 2004: image processing, vol 5370, pp 304–313. International Society for Optics and Photonics

  44. Ino Fumihiko, Ooyama Kanrou, Hagihara Kenichi (2005) A data distributed parallel algorithm for nonrigid image registration. Parallel Comput 31(1):19–43

    Article  Google Scholar 

Download references

Acknowledgements

This project is co-financed with tax funds on the basis of the budget approved by the Saxon state parliament.

Author information

Authors and Affiliations

  1. Institute of Circuits and Systems, Faculty of Electrical and Computer Engineering, Technische Universität Dresden, 01062, Dresden, Germany

    Yahya Moshaei-Nezhad & Ronald Tetzlaff

  2. Carl Gustav Carus Faculty of Medicine, Anesthesiology and Intensive Care Medicine, Clinical Sensoring and Monitoring, Technische Universität Dresden, Fetscherstraße 74, 01307, Dresden, Germany

    Juliane Müller & Martin Oelschlägel

  3. Carl Gustav Carus Faculty of Medicine, Department of Neurosurgery, Technische Universität Dresden, Fetscherstraße 74, 01307, Dresden, Germany

    Matthias Kirsch

  4. Department of Neurosurgery, Asklepios Kliniken Schildautal, Karl-Herold-Str. 1, 38723, Seesen, Germany

    Matthias Kirsch

Authors
  1. Yahya Moshaei-Nezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juliane Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Oelschlägel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matthias Kirsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ronald Tetzlaff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yahya Moshaei-Nezhad.

Ethics declarations

Conflict of interest

Authors state no conflict of interest.

Ethical approval

The research related to the human use complies with all the relevant national regulations, institutional policies and was performed in accordance with the tenets of the Helsinki Declaration

Informed consent

Informed consent has been obtained from all patients.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moshaei-Nezhad, Y., Müller, J., Oelschlägel, M. et al. Registration of IRT and visible light images in neurosurgery: analysis and comparison of automatic intensity-based registration approaches. Int J CARS 17, 683–697 (2022). https://doi.org/10.1007/s11548-022-02562-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02562-x

Keywords

Search

Navigation