Multimodal image registration based on binary gradient angle descriptor

  • Dongsheng Jiang
  • Yonghong Shi
  • Demin Yao
  • Yifeng Fan
  • Manning WangEmail author
  • Zhijian SongEmail author
Original Article



Multimodal image registration plays an important role in image-guided interventions/therapy and atlas building, and it is still a challenging task due to the complex intensity variations in different modalities.


The paper addresses the problem and proposes a simple, compact, fast and generally applicable modality-independent binary gradient angle descriptor (BGA) based on the rationale of gradient orientation alignment. The BGA can be easily calculated at each voxel by coding the quadrant in which a local gradient vector falls, and it has an extremely low computational complexity, requiring only three convolutions, two multiplication operations and two comparison operations. Meanwhile, the binarized encoding of the gradient orientation makes the BGA more resistant to image degradations compared with conventional gradient orientation methods. The BGA can extract similar feature descriptors for different modalities and enable the use of simple similarity measures, which makes it applicable within a wide range of optimization frameworks.


The results for pairwise multimodal and monomodal registrations between various images (T1, T2, PD, T1c, Flair) consistently show that the BGA significantly outperforms localized mutual information. The experimental results also confirm that the BGA can be a reliable alternative to the sum of absolute difference in monomodal image registration. The BGA can also achieve an accuracy of \(1.41\pm 1.1~\hbox {mm}\), similar to that of the SSC, for the deformable registration of inhale and exhale CT scans. Specifically, for the highly challenging deformable registration of preoperative MRI and 3D intraoperative ultrasound images, the BGA achieves a similar registration accuracy of \(2.33\pm 1.2~\hbox {mm}\) compared with state-of-the-art approaches, with a computation time of 18.3 s per case.


The BGA improves the registration performance in terms of both accuracy and time efficiency. With further acceleration, the framework has the potential for application in time-sensitive clinical environments, such as for preoperative MRI and intraoperative US image registration for image-guided intervention.


Binary gradient angle descriptor Multimodal image registration Intrasubject image registration Hamming distance 



The authors would like to thank Dr. Mattias P. Heinrich for his help in the implementation of his discrete optimization.

Funding This study has been supported by: the Nation Natural Science Foundation of China (Projects 81271670 and 60972102), the National Science and Technology Support Program (No. 2015BAK31B01), the National High Technology Research and Development Program (2015AA020507), Science and Technology Commission of Shanghai Municipality (No. 15441905500) and Zhejiang Public Technology Research Program (No. 2016C33120).

Compliance with ethical standards

Conflict of interest

The authors have no relevant conflict of interest to disclose.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Statement of informed consent was not applicable since the manuscript does not contain any participants’ data.


  1. 1.
    Maybody M, Stevenson C, Solomon SB (2013) Overview of navigation systems in image-guided interventions. Tech Vasc Interv Radiol 16:136–143CrossRefPubMedGoogle Scholar
  2. 2.
    Liao R, Zhang L, Sun Y, Miao S, Chefd’Hotel C (2013) A review of recent advances in registration techniques applied to minimally invasive therapy. IEEE Trans Multimed 15:983–1000CrossRefGoogle Scholar
  3. 3.
    Ewertsen C, Săftoiu A, Gruionu LG, Karstrup S, Nielsen MB (2013) Real-time image fusion involving diagnostic ultrasound. Am J Roentgenol 200:W249–W255CrossRefGoogle Scholar
  4. 4.
    Wu G, Jia H, Wang Q, Shen D (2011) SharpMean: groupwise registration guided by sharp mean image and tree-based registration. Neuroimage 56:1968–1981CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Glocker B, Sotiras A, Komodakis N, Paragios N (2011) Deformable medical image registration: setting the state of the art with discrete methods. Annu Rev Biomed Eng 13:219–244CrossRefPubMedGoogle Scholar
  6. 6.
    Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16:187–198CrossRefPubMedGoogle Scholar
  7. 7.
    Haber E, Modersitzki J (2005) Beyond mutual information: a simple and robust alternative. Bildverarbeitung für die Medizin 2005:350–354Google Scholar
  8. 8.
    De Nigris D, Collins DL, Arbel T (2012) Multi-modal image registration based on gradient orientations of minimal uncertainty. IEEE Trans Med Imaging 31:2343–2354CrossRefPubMedGoogle Scholar
  9. 9.
    Heinrich M, Schnabel J, Gleeson F, Brady M, Jenkinson M (2010) Non-rigid multimodal medical image registration using optical flow and gradient orientation. In: Proceedings of medical image analysis and understanding, pp 141–145Google Scholar
  10. 10.
    Heinrich MP, Jenkinson M, Gleeson FV, Brady SM, Schnabel JA (2011) Deformable multimodal registration with gradient orientation based on structure tensors. Ann Br Mach Vis Assoc 5:1–11Google Scholar
  11. 11.
    König L, Rühaak J (2014) A fast and accurate parallel algorithm for non-linear image registration using normalized gradient fields. In: 2014 IEEE 11th international symposium on biomedical imaging (ISBI). IEEE, pp 580–583Google Scholar
  12. 12.
    Wachinger C, Navab N (2012) Entropy and laplacian images: structural representations for multi-modal registration. Med Image Anal 16:1–17CrossRefPubMedGoogle Scholar
  13. 13.
    Yang F, Ding M, Zhang X, Wu Y, Hu J (2013) Two phase non-rigid multi-modal image registration using Weber local descriptor-based similarity metrics and normalized mutual information. Sensors (Basel) 13:7599–7617CrossRefGoogle Scholar
  14. 14.
    Piella Fenoy G (2014) Diffusion maps for multimodal registration. Sensors. 14(6):10563–10577Google Scholar
  15. 15.
    Heinrich MP, Jenkinson M, Bhushan M, Matin T, Gleeson FV, Brady SM, Schnabel JA (2012) MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Med Image Anal 16:1423–1435CrossRefPubMedGoogle Scholar
  16. 16.
    Heinrich MP, Jenkinson M, Papież BW, Brady M, Schnabel JA (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: Medical image computing and computer-assisted intervention–MICCAI 2013. Springer, pp 187–194Google Scholar
  17. 17.
    Heinrich MP, Maier O, Handels H (2015) Multi-modal multi-Atlas segmentation using discrete optimisation and self-similarities. VISCERAL@ ISBI 2015 VISCERAL Anatomy3 Organ Segmentation Challenge 27Google Scholar
  18. 18.
    Jiang D, Shi Y, Yao D, Wang M, Song Z (2016) miLBP: a robust and fast modality-independent 3D LBP for multimodal deformable registration. Int J Comput Assist Radiol 11:997–1005Google Scholar
  19. 19.
    Li Z, Mahapatra D, Tielbeek JA, Stoker J, van Vliet LJ, Vos FM (2016) Image registration based on autocorrelation of local structure. IEEE Trans Med Imaging 35:63–75CrossRefPubMedGoogle Scholar
  20. 20.
    Toews M, Zöllei L, Wells III WM (2013) Feature-based alignment of volumetric multi-modal images. In: Information processing in medical imaging: proceedings of the... conference NIH Public Access, p 25Google Scholar
  21. 21.
    Haber E, Modersitzki J (2006) Intensity gradient based registration and fusion of multi-modal images. In: Medical image computing and computer-assisted intervention—MICCAI 2006. Springer, pp 726–733Google Scholar
  22. 22.
    Heinrich MP, Jenkinson M, Brady M, Schnabel JA (2013) MRF-based deformable registration and ventilation estimation of lung CT. IEEE Trans Med Imaging 32:1239–1248CrossRefPubMedGoogle Scholar
  23. 23.
    Heinrich MP, Simpson IJA, Papież BW, Brady SM, Schnabel JA (2016) Deformable image registration by combining uncertainty estimates from supervoxel belief propagation. Med Image Anal 27:57–71CrossRefPubMedGoogle Scholar
  24. 24.
    Castillo R, Castillo E, Guerra R, Johnson VE, McPhail T, Garg AK, Guerrero T (2009) A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys Med Biol 54:1849CrossRefPubMedGoogle Scholar
  25. 25.
    Fuerst B, Wein W, Müller M, Navab N (2014) Automatic ultrasound—MRI registration for neurosurgery using the 2D and 3D LC\(^2\) Metric. Med Image Anal 18(8):1312–1319CrossRefPubMedGoogle Scholar
  26. 26.
    Rivaz H, Karimaghaloo Z, Collins DL (2014) Self-similarity weighted mutual information: a new nonrigid image registration metric. Med Image Anal 18:343–358CrossRefPubMedGoogle Scholar
  27. 27.
    Rivaz H, Karimaghaloo Z, Fonov VS, Collins DL (2014) Nonrigid registration of ultrasound and MRI using contextual conditioned mutual information. IEEE Trans Med Imaging 33(3):708–725CrossRefPubMedGoogle Scholar
  28. 28.
    Avants BB, Epstein CL, Grossman M, Gee JC (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12:26–41CrossRefPubMedGoogle Scholar
  29. 29.
    Jiang D, Shi Y, Chen X, Wang M, Song Z (2016) Fast and robust multimodal image registration using a local derivative pattern. Med Phys 44(2):497–509CrossRefGoogle Scholar

Copyright information

© CARS 2017

Authors and Affiliations

  1. 1.Digital Medical Research Center, School of Basic Medical ScienceFudan UniversityShanghaiChina
  2. 2.Shanghai Key Laboratory of Medical Image Computing and Computer-Assisted InterventionShanghaiChina
  3. 3.School of Medical ImagingHangzhou Medical CollegeHangzhouChina

Personalised recommendations