Skip to main content

Advertisement

SpringerLink
Log in

Multimodal image registration based on binary gradient angle descriptor

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Multimodal image registration plays an important role in image-guided interventions/therapy and atlas building, and it is still a challenging task due to the complex intensity variations in different modalities.

Methods

The paper addresses the problem and proposes a simple, compact, fast and generally applicable modality-independent binary gradient angle descriptor (BGA) based on the rationale of gradient orientation alignment. The BGA can be easily calculated at each voxel by coding the quadrant in which a local gradient vector falls, and it has an extremely low computational complexity, requiring only three convolutions, two multiplication operations and two comparison operations. Meanwhile, the binarized encoding of the gradient orientation makes the BGA more resistant to image degradations compared with conventional gradient orientation methods. The BGA can extract similar feature descriptors for different modalities and enable the use of simple similarity measures, which makes it applicable within a wide range of optimization frameworks.

Results

The results for pairwise multimodal and monomodal registrations between various images (T1, T2, PD, T1c, Flair) consistently show that the BGA significantly outperforms localized mutual information. The experimental results also confirm that the BGA can be a reliable alternative to the sum of absolute difference in monomodal image registration. The BGA can also achieve an accuracy of \(1.41\pm 1.1~\hbox {mm}\), similar to that of the SSC, for the deformable registration of inhale and exhale CT scans. Specifically, for the highly challenging deformable registration of preoperative MRI and 3D intraoperative ultrasound images, the BGA achieves a similar registration accuracy of \(2.33\pm 1.2~\hbox {mm}\) compared with state-of-the-art approaches, with a computation time of 18.3 s per case.

Conclusions

The BGA improves the registration performance in terms of both accuracy and time efficiency. With further acceleration, the framework has the potential for application in time-sensitive clinical environments, such as for preoperative MRI and intraoperative US image registration for image-guided intervention.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Maybody M, Stevenson C, Solomon SB (2013) Overview of navigation systems in image-guided interventions. Tech Vasc Interv Radiol 16:136–143

    Article  PubMed  Google Scholar 

  2. Liao R, Zhang L, Sun Y, Miao S, Chefd’Hotel C (2013) A review of recent advances in registration techniques applied to minimally invasive therapy. IEEE Trans Multimed 15:983–1000

    Article  Google Scholar 

  3. Ewertsen C, Săftoiu A, Gruionu LG, Karstrup S, Nielsen MB (2013) Real-time image fusion involving diagnostic ultrasound. Am J Roentgenol 200:W249–W255

    Article  Google Scholar 

  4. Wu G, Jia H, Wang Q, Shen D (2011) SharpMean: groupwise registration guided by sharp mean image and tree-based registration. Neuroimage 56:1968–1981

    Article  PubMed  PubMed Central  Google Scholar 

  5. Glocker B, Sotiras A, Komodakis N, Paragios N (2011) Deformable medical image registration: setting the state of the art with discrete methods. Annu Rev Biomed Eng 13:219–244

    Article  CAS  PubMed  Google Scholar 

  6. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16:187–198

    Article  CAS  PubMed  Google Scholar 

  7. Haber E, Modersitzki J (2005) Beyond mutual information: a simple and robust alternative. Bildverarbeitung für die Medizin 2005:350–354

    Google Scholar 

  8. De Nigris D, Collins DL, Arbel T (2012) Multi-modal image registration based on gradient orientations of minimal uncertainty. IEEE Trans Med Imaging 31:2343–2354

    Article  PubMed  Google Scholar 

  9. Heinrich M, Schnabel J, Gleeson F, Brady M, Jenkinson M (2010) Non-rigid multimodal medical image registration using optical flow and gradient orientation. In: Proceedings of medical image analysis and understanding, pp 141–145

  10. Heinrich MP, Jenkinson M, Gleeson FV, Brady SM, Schnabel JA (2011) Deformable multimodal registration with gradient orientation based on structure tensors. Ann Br Mach Vis Assoc 5:1–11

    Google Scholar 

  11. König L, Rühaak J (2014) A fast and accurate parallel algorithm for non-linear image registration using normalized gradient fields. In: 2014 IEEE 11th international symposium on biomedical imaging (ISBI). IEEE, pp 580–583

  12. Wachinger C, Navab N (2012) Entropy and laplacian images: structural representations for multi-modal registration. Med Image Anal 16:1–17

    Article  PubMed  Google Scholar 

  13. Yang F, Ding M, Zhang X, Wu Y, Hu J (2013) Two phase non-rigid multi-modal image registration using Weber local descriptor-based similarity metrics and normalized mutual information. Sensors (Basel) 13:7599–7617

    Article  Google Scholar 

  14. Piella Fenoy G (2014) Diffusion maps for multimodal registration. Sensors. 14(6):10563–10577

    Google Scholar 

  15. Heinrich MP, Jenkinson M, Bhushan M, Matin T, Gleeson FV, Brady SM, Schnabel JA (2012) MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Med Image Anal 16:1423–1435

    Article  PubMed  Google Scholar 

  16. Heinrich MP, Jenkinson M, Papież BW, Brady M, Schnabel JA (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: Medical image computing and computer-assisted intervention–MICCAI 2013. Springer, pp 187–194

  17. Heinrich MP, Maier O, Handels H (2015) Multi-modal multi-Atlas segmentation using discrete optimisation and self-similarities. VISCERAL@ ISBI 2015 VISCERAL Anatomy3 Organ Segmentation Challenge 27

  18. Jiang D, Shi Y, Yao D, Wang M, Song Z (2016) miLBP: a robust and fast modality-independent 3D LBP for multimodal deformable registration. Int J Comput Assist Radiol 11:997–1005

  19. Li Z, Mahapatra D, Tielbeek JA, Stoker J, van Vliet LJ, Vos FM (2016) Image registration based on autocorrelation of local structure. IEEE Trans Med Imaging 35:63–75

    Article  CAS  PubMed  Google Scholar 

  20. Toews M, Zöllei L, Wells III WM (2013) Feature-based alignment of volumetric multi-modal images. In: Information processing in medical imaging: proceedings of the... conference NIH Public Access, p 25

  21. Haber E, Modersitzki J (2006) Intensity gradient based registration and fusion of multi-modal images. In: Medical image computing and computer-assisted intervention—MICCAI 2006. Springer, pp 726–733

  22. Heinrich MP, Jenkinson M, Brady M, Schnabel JA (2013) MRF-based deformable registration and ventilation estimation of lung CT. IEEE Trans Med Imaging 32:1239–1248

    Article  PubMed  Google Scholar 

  23. Heinrich MP, Simpson IJA, Papież BW, Brady SM, Schnabel JA (2016) Deformable image registration by combining uncertainty estimates from supervoxel belief propagation. Med Image Anal 27:57–71

    Article  PubMed  Google Scholar 

  24. Castillo R, Castillo E, Guerra R, Johnson VE, McPhail T, Garg AK, Guerrero T (2009) A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys Med Biol 54:1849

    Article  PubMed  Google Scholar 

  25. Fuerst B, Wein W, Müller M, Navab N (2014) Automatic ultrasound—MRI registration for neurosurgery using the 2D and 3D LC\(^2\) Metric. Med Image Anal 18(8):1312–1319

    Article  PubMed  Google Scholar 

  26. Rivaz H, Karimaghaloo Z, Collins DL (2014) Self-similarity weighted mutual information: a new nonrigid image registration metric. Med Image Anal 18:343–358

    Article  PubMed  Google Scholar 

  27. Rivaz H, Karimaghaloo Z, Fonov VS, Collins DL (2014) Nonrigid registration of ultrasound and MRI using contextual conditioned mutual information. IEEE Trans Med Imaging 33(3):708–725

    Article  PubMed  Google Scholar 

  28. Avants BB, Epstein CL, Grossman M, Gee JC (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12:26–41

    Article  CAS  PubMed  Google Scholar 

  29. Jiang D, Shi Y, Chen X, Wang M, Song Z (2016) Fast and robust multimodal image registration using a local derivative pattern. Med Phys 44(2):497–509

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Mattias P. Heinrich for his help in the implementation of his discrete optimization.

Funding This study has been supported by: the Nation Natural Science Foundation of China (Projects 81271670 and 60972102), the National Science and Technology Support Program (No. 2015BAK31B01), the National High Technology Research and Development Program (2015AA020507), Science and Technology Commission of Shanghai Municipality (No. 15441905500) and Zhejiang Public Technology Research Program (No. 2016C33120).

Author information

Author notes

  1. Dongsheng Jiang and Yonghong Shi are co-first authors, and they have contributed equally to this work.

Authors and Affiliations

  1. Digital Medical Research Center, School of Basic Medical Science, Fudan University, Shanghai, 200032, China

    Dongsheng Jiang, Yonghong Shi, Demin Yao, Manning Wang & Zhijian Song

  2. Shanghai Key Laboratory of Medical Image Computing and Computer-Assisted Intervention, Shanghai, 200032, China

    Dongsheng Jiang, Yonghong Shi, Demin Yao, Manning Wang & Zhijian Song

  3. School of Medical Imaging, Hangzhou Medical College, Hangzhou, 310053, China

    Yifeng Fan

Authors
  1. Dongsheng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yonghong Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Demin Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yifeng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Manning Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhijian Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Manning Wang or Zhijian Song.

Ethics declarations

Conflict of interest

The authors have no relevant conflict of interest to disclose.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Statement of informed consent was not applicable since the manuscript does not contain any participants’ data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, D., Shi, Y., Yao, D. et al. Multimodal image registration based on binary gradient angle descriptor. Int J CARS 12, 2157–2167 (2017). https://doi.org/10.1007/s11548-017-1661-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-017-1661-y

Keywords

Search

Navigation