Advertisement

ARENA: Inter-modality affine registration using evolutionary strategy

  • Nima MasoumiEmail author
  • Yiming Xiao
  • Hassan Rivaz
Original Article

Abstract

Purpose

Image fusion of different imaging modalities renders valuable information to clinicians. In this paper, we proposed an automatic multimodal registration method to register intra-operative ultrasound images (US) to preoperative magnetic resonance images (MRI) in the context of image-guided neurosurgery.

Methods

We employed refined correlation ratio as a similarity metric for our intensity-based image registration method. We deem MRI as the fixed image (\(I_\mathrm{f}\)) and US as the moving image (\(I_\mathrm{m}\)) and then transform \(I_\mathrm{m}\) to align with \(I_\mathrm{f}\). We utilized the covariance matrix adaptation evolutionary strategy to find the optimal affine transformation in registration of \(I_\mathrm{m}\) to \(I_\mathrm{f}\).

Results

We applied our method on the publicly available retrospective evaluation of cerebral tumors (RESECT) database and Montreal Neurological Institute’s brain images of tumors for evaluation (BITE) database. We validated the results qualitatively and quantitatively. Qualitative validation is conducted (by the three authors) through overlaying pre- and post-registration US and MRI to allow visual assessment of the alignment. Quantitative validation is performed by utilizing the corresponding landmarks in the databases for the preoperative MRI and the intra-operative US. Average mean target registration error (mTRE) has been reduced from \(5.40\pm 4.27\) to \(2.77\pm 1.13\) in 22 patients in the RESECT database and from \(4.12\pm 2.03\) to \(2.82\pm 0.72\) in the BITE database. A nonparametric statistical analysis performed using the Wilcoxon rank sum test shows that there is a significant difference between pre- and post-registration mTREs with a p value of \(0.0058\,(p<0.05)\) for the RESECT database and \(0.0483\,(p<0.05)\) for the BITE database.

Conclusions

The proposed fully automatic registration method significantly improved the alignment of MRI and US images and can therefore be used to reduce the misalignment of US and MRI caused by brain shift, calibration errors, and patient to MRI transformation matrix.

Keywords

Image registration Correlation ratio Affine Transformation CMA-ES RESECT database mTRE 

Notes

Acknowledgements

This work is funded by Natural Science Engineering Council of Canada (NSERC) Grant RGPIN-2015-04136.

Compliance with ethical standards

Conflict of interest

The authors declare that there is no conflict of interest.

Ethical standard

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all participants included in the study.

References

  1. 1.
    Damas S, Cordón O, Santamaría J (2011) Medical image registration using evolutionary computation: an experimental survey. IEEE Comput Intell Mag 6(4):26–42CrossRefGoogle Scholar
  2. 2.
    Ma J, Zhou H, Zhao J, Gao Y, Jiang J, Tian J (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481CrossRefGoogle Scholar
  3. 3.
    James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fusion 19:4–19CrossRefGoogle Scholar
  4. 4.
    Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33:100–112CrossRefGoogle Scholar
  5. 5.
    Yang Y, Que Y, Huang S, Lin P (2016) Multimodal sensor medical image fusion based on type-2 fuzzy logic in nsct domain. IEEE Sens J 16(10):3735–3745CrossRefGoogle Scholar
  6. 6.
    Golby AJ (2015) Image-guided neurosurgery. Academic Press, CambridgeGoogle Scholar
  7. 7.
    Besharati Tabrizi L, Mahvash M (2015) Augmented reality–guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123(1):206–211CrossRefPubMedGoogle Scholar
  8. 8.
    Maurer CR, Fitzpatrick JM (1993) A review of medical image registration. Interact Image Guid Neurosurg 1:17–44Google Scholar
  9. 9.
    Gerard IJ, Kersten-Oertel M, Petrecca K, Sirhan D, Hall JA, Collins DL (2017) Brain shift in neuronavigation of brain tumors: a review. Med Image Anal 35:403–420CrossRefPubMedGoogle Scholar
  10. 10.
    Nag S (2017) Image registration techniques: a survey. arXiv preprint arXiv:1712.07540
  11. 11.
    Maintz JA, Viergever MA (1998) A survey of medical image registration. Med Image Anal 2(1):1–36CrossRefPubMedGoogle Scholar
  12. 12.
    Gong M, Zhao S, Jiao L, Tian D, Wang S (2014) A novel coarse-to-fine scheme for automatic image registration based on sift and mutual information. IEEE Trans Geosci Remote Sens 52(7):4328–4338CrossRefGoogle Scholar
  13. 13.
    Johnson HJ, Christensen GE (2002) Consistent landmark and intensity-based image registration. IEEE Trans Med Imaging 21(5):450–461CrossRefPubMedGoogle Scholar
  14. 14.
    Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000CrossRefGoogle Scholar
  15. 15.
    Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40(1):16–28CrossRefGoogle Scholar
  16. 16.
    Rueckert D, Aljabar P (2010) Nonrigid registration of medical images: theory, methods, and applications [applications corner]. IEEE Signal Process Mag 27(4):113–119CrossRefGoogle Scholar
  17. 17.
    Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: a survey. IEEE Trans Med Imaging 32(7):1153–1190CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Yan CX, Goulet B, Pelletier J, Chen SJ-S, Tampieri D, Collins DL (2011) Towards accurate, robust and practical ultrasound-ct registration of vertebrae for image-guided spine surgery. Int J Comput Assist Radiol Surg 6(4):523–537CrossRefPubMedGoogle Scholar
  19. 19.
    Gill S, Abolmaesumi P, Fichtinger G, Boisvert J, Pichora D, Borshneck D, Mousavi P (2012) Biomechanically constrained groupwise ultrasound to ct registration of the lumbar spine. Med Image Anal 16(3):662–674CrossRefPubMedGoogle Scholar
  20. 20.
    Hacihaliloglu I, Rasoulian A, Rohling RN, Abolmaesumi P (2014) Local phase tensor features for 3-d ultrasound to statistical shape+ pose spine model registration. IEEE Trans Med Imaging 33(11):2167–2179CrossRefPubMedGoogle Scholar
  21. 21.
    Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2018) An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9252–9260Google Scholar
  22. 22.
    Weistrand O, Svensson S (2015) The anaconda algorithm for deformable image registration in radiotherapy. Med Phys 42(1):40–53CrossRefPubMedGoogle Scholar
  23. 23.
    Zhao B, Christensen GE, Hyun Song J, Pan Y, Gerard SE, Reinhardt JM, Du K, Patton T, Bayouth JM, Hugo GD (2016) Tissue-volume preserving deformable image registration for 4dct pulmonary images. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 41–49Google Scholar
  24. 24.
    Maes F, Loeckx D, Vandermeulen D, Suetens P (2015) Image registration using mutual information. In: Paragios N, Duncan J, Ayache N (eds) Handbook of biomedical imaging. Springer, Boston, MAGoogle Scholar
  25. 25.
    Roche A, Malandain G, Ayache N, Pennec X (1998) Multimodal image registration by maximization of the correlation ratio. PhD thesis, INRIAGoogle Scholar
  26. 26.
    Roche A, Pennec X, Rudolph M, Auer D, Malandain G, Ourselin S, Auer LM, Ayache N (2000) Generalized correlation ratio for rigid registration of 3d ultrasound with mr images. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 567–577Google Scholar
  27. 27.
    Rivaz H, Collins DL (2015) Deformable registration of preoperative mr, pre-resection ultrasound, and post-resection ultrasound images of neurosurgery. Int J Comput Assist Radiol Surg 10(7):1017–1028CrossRefPubMedGoogle Scholar
  28. 28.
    Masoumi N, Xiao Y, Rivaz H (2017) Marcel (inter-modality affine registration with correlation ratio): an application for brain shift correction in ultrasound-guided brain tumor resection. In: International MICCAI Brainlesion workshop. Springer, pp 55–63Google Scholar
  29. 29.
    Rivaz H, Chen SJ-S, Collins DL (2015) Automatic deformable mr-ultrasound registration for image-guided neurosurgery. IEEE Trans Med Imaging 34(2):366–380CrossRefPubMedGoogle Scholar
  30. 30.
    Hansen N, Ostermeier A (1996) Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation. In: Evolutionary computation, 1996., Proceedings of IEEE international conference on. IEEE, pp 312–317Google Scholar
  31. 31.
    Xiao Y, Fortin M, Unsgård G, Rivaz H, Reinertsen I (2017) Retrospective evaluation of cerebral tumors (resect): a clinical database of pre-operative mri and intra-operative ultrasound in low-grade glioma surgeries. Med Phys 44:3875–3882CrossRefPubMedGoogle Scholar
  32. 32.
    Machado I, Toews M, Luo J, Unadkat P, Essayed W, George E, Teodoro P, Carvalho H, Martins J, Golland P, Pieper S (2018) Non-rigid registration of 3d ultrasound for neurosurgery using automatic feature detection and matching. Int J Comput Assist Radiol Surg 13:1525–1538CrossRefPubMedGoogle Scholar
  33. 33.
    Mercier L, Del Maestro RF, Petrecca K, Araujo D, Haegelen C, Collins DL (2012) Online database of clinical mr and ultrasound images of brain tumors. Med Phys 39(6 Part1):3253–3261CrossRefPubMedGoogle Scholar
  34. 34.
    Klein S, Staring M, Pluim JP (2007) Evaluation of optimization methods for nonrigid medical image registration using mutual information and b-splines. IEEE Trans Image Process 16(12):2879–2890CrossRefPubMedGoogle Scholar
  35. 35.
    Winter S, Brendel B, Pechlivanis I, Schmieder K, Igel C (2008) Registration of ct and intraoperative 3-d ultrasound images of the spine using evolutionary and gradient-based methods. IEEE Trans Evol Comput 12(3):284–296CrossRefGoogle Scholar
  36. 36.
    Gong RH, Abolmaesumi P (2008) 2d/3d registration with the cma-es method. In: Medical imaging 2008: visualization, image-guided procedures, and modeling. International Society for Optics and Photonics, vol 6918, p 69181MGoogle Scholar
  37. 37.
    Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, Taylor RH (2012) Intraoperative image-based multiview 2d/3d registration for image-guided orthopaedic surgery: incorporation of fiducial-based c-arm tracking and gpu-acceleration. IEEE Trans Med Imaging 31(4):948–962CrossRefPubMedGoogle Scholar
  38. 38.
    Reinhard E, Heidrich W, Debevec P, Pattanaik S, Ward G, Myszkowski K (2010) High dynamic range imaging: acquisition, display, and image-based lighting. Morgan Kaufmann, BurlingtonGoogle Scholar
  39. 39.
    Fischer B, Modersitzki J (2008) Ill-posed medicine–an introduction to image registration. Inverse Probl 24(3):034008CrossRefGoogle Scholar
  40. 40.
    Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195CrossRefPubMedGoogle Scholar
  41. 41.
    “Cma-es in matlab-yarpiz”Google Scholar
  42. 42.
    Wein W, Ladikos A, Fuerst B, Shah A, Sharma K, Navab N (2013) Global registration of ultrasound to mri using the lc 2 metric for enabling neurosurgical guidance. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 34–41Google Scholar
  43. 43.
    Heinrich MP, Jenkinson M, Papież BW, Brady M, Schnabel JA (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 187–194Google Scholar
  44. 44.
    Daga P, Winston G, Modat M, White M, Mancini L, Cardoso MJ, Symms M, Stretton J, McEvoy AW, Thornton J, Micallef C (2012) Accurate localization of optic radiation during neurosurgery in an interventional mri suite. IEEE Trans Med Imaging 31(4):882–891CrossRefPubMedGoogle Scholar
  45. 45.
    Fitzpatrick JM (2009) Fiducial registration error and target registration error are uncorrelated. In: Medical imaging 2009: visualization, image-guided procedures, and modeling, International Society for Optics and Photonics, vol 7261, p 726102 (2009)Google Scholar
  46. 46.
    Zhong X, Bayer S, Ravikumar N, Strobel N, Birkhold A, Kowarschik M, Fahrig R, Maier A (2018) Resolve intraoperative brain shift as imitation game. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 129–137Google Scholar
  47. 47.
    Hong J, Park H (2018) Non-linear approach for mri to intra-operative us registration using structural skeleton. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 138–145Google Scholar
  48. 48.
    Wein W (2018) Brain-shift correction with image-based registration and landmark accuracy evaluation. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 146–151Google Scholar
  49. 49.
    Sun L, Zhang S (2018) Deformable mri-ultrasound registration using 3d convolutional neural network. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 152–158Google Scholar
  50. 50.
    Heinrich MP (2018) Intra-operative ultrasound to mri fusion with a public multimodal discrete registration tool. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 159–164Google Scholar
  51. 51.
    Machado I, Toews M, Luo J, Unadkat P, Essayed W, George E, Teodoro P, Carvalho H, Martins J, Golland P (2018) Deformable mri-ultrasound registration via attribute matching and mutual-saliency weighting for image-guided neurosurgery. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 165–171Google Scholar
  52. 52.
    Drobny D, Vercauteren T, Ourselin S, Modat M (2018) Registration of mri and ius data to compensate brain shift using a symmetric block-matching based approach. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 172–178Google Scholar
  53. 53.
    Shams R, Boucher M-A, Kadoury S (2018) Intra-operative brain shift correction with weighted locally linear correlations of 3dus and mri. In: Simulation, image processing, and ultrasound systems for assisted diagnosis and navigation. Springer, pp 179–184Google Scholar
  54. 54.
    Gibbons JD, Chakraborti S (2011) Nonparametric statistical inference. In: Lovric M (ed) International encyclopedia of statistical science. Springer, Berlin, HeidelbergGoogle Scholar
  55. 55.
    Avants BB, Tustison NJ, Song G, Cook PA, Klein A, Gee JC (2011) A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3):2033–2044CrossRefPubMedGoogle Scholar
  56. 56.
    Modat M, Cardoso MJ, Daga P, Cash D, Fox NC, Ourselin S (2012) Inverse-consistent symmetric free form deformation. In: International workshop on biomedical image registration. Springer, pp 79–88Google Scholar
  57. 57.
    Xiao Y, Eikenes L, Reinertsen I, Rivaz H (2018) Nonlinear deformation of tractography in ultrasound-guided low-grade gliomas resection. Int J Comput Assist Radiol Surg 13(3):457–467CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2018

Authors and Affiliations

  1. 1.PERFORM CentreConcordia UniversityMontrealCanada
  2. 2.Department of Electrical and Computer EngineeringConcordia UniversityMontrealCanada
  3. 3.Robarts Research InstituteWestern UniversityLondonCanada

Personalised recommendations