Multimodal image registration with joint structure tensor and local entropy

  • Jingya Zhang
  • Jiajun WangEmail author
  • Xiuying Wang
  • Dagan Feng
Original Article



Nonrigid registration of multimodal medical images remains a challenge in image-guided interventions. A common approach is to use mutual information (MI), which is robust to the intensity variations across modalities. However, primarily based on intensity distribution, MI does not take into account of underlying spatial and structural information of the images, which might lead to local optimization. To address such a challenge, this paper proposes a two-stage multimodal nonrigid registration scheme with joint structural information and local entropy.


In our two-stage multimodal nonrigid registration scheme, both the reference image and floating image are firstly converted to a common space. A unified representation in the common space for the images is constructed by fusing the structure tensor (ST) trace with the local entropy (LE). Through the representation that reflects its geometry uniformly across modalities, the complicated deformation field is estimated using \(L_{1}\) or \(L_{2}\) distance.


We compared our approach to four other methods: (1) the method using LE, (2) the method using ST, (3) the method using spatially weighted LE and (4) the conventional MI-based method. Quantitative evaluations on 80 multimodal image pairs of different organs including 50 pairs of MR images with artificial deformations, 20 pairs of medical brain MR images and 10 pairs of breast images showed that our proposed method outperformed the comparison methods. Student’s t test demonstrated that our method achieved statistically significant improvement on registration accuracy.


The two-stage registration with joint ST and LE outperformed the conventional MI-based method for multimodal images. Both the ST and the LE contributed to the improved registration accuracy.


Nonrigid registration Structure tensor Local entropy Mutual information 



This work is supported by National Natural Science Foundation of China, No. 60871086 and Natural Science Foundation of Jiangsu Province China, No. BK2008159.

Conflict of interest



  1. 1.
    Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: a survey. IEEE Trans Med Imaging 32(7):1153–1190PubMedCentralCrossRefPubMedGoogle Scholar
  2. 2.
    Markelj P, Tomaževič D, Likar B, Pernuš F (2012) A review of 3D/2D registration methods for image-guided interventions. Med Image Anal 16(3):642–661CrossRefPubMedGoogle Scholar
  3. 3.
    Roujol S, Benois-Pineau J, de Senneville BD, Ries M, Quesson B, Moonen CTW (2012) Robust real-time-constrained estimation of respiratory motion for interventional MRI on mobile organs. IEEE Trans Inf Technol Biomed 16(3):365–374CrossRefPubMedGoogle Scholar
  4. 4.
    Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, Taylor RH (2012) Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU-acceleration. IEEE Trans Med Imaging 31(4):948–962PubMedCentralCrossRefPubMedGoogle Scholar
  5. 5.
    Ferrant M, Nabavi A, Macq B, Black P, Jolesz F, Kikinis R, Warfield S (2012) Serial registration of intraoperative MR images of the brain. Med Image Anal 6(4):337–360CrossRefGoogle Scholar
  6. 6.
    Wang X, Eberl S, Fulham M, Som S, Feng DD (2008) Data registration and fusion. In: Feng DD (ed) Biomedical information technology. Elsevier, Amsterdam, pp 187–210Google Scholar
  7. 7.
    Lu X, Zhao Y, Zhang B, Wu J, Li N, Jia W (2013) A non-rigid cardiac image registration method based on an optical flow model. Opt Int J Light Electron Opt 124(20):4266–4273CrossRefGoogle Scholar
  8. 8.
    Lombaert H, Grady L, Pennec X, Ayache N, Cheriet F (2014) Spectral log-demons: diffeomorphic image registration with very large deformations. Int J Comput Vis 107(3):254–271CrossRefGoogle Scholar
  9. 9.
    Zhang J, Wang J, Wang X, Feng D (2014) The adaptive FEM elastic model for medical image registration. Phys Med Biol 59:97–118CrossRefPubMedGoogle Scholar
  10. 10.
    Zhuang X, Arridge S, Hawkes DJ, Ourselin S (2011) A nonrigid registration framework using spatially encoded mutual information and free-form deformations. IEEE Trans Med Imaging 30(10):1819–1828CrossRefPubMedGoogle Scholar
  11. 11.
    Wachinger C, Navab N (2012) Entropy and Laplacian images: structural representations for multi-modal registration. Med Image Anal 16(1):1–17CrossRefPubMedGoogle Scholar
  12. 12.
    Budde MD, Frank JA (2012) Examining brain microstructure using structure tensor analysis of histological sections. Neuroimage 63(1):1–10CrossRefPubMedGoogle Scholar
  13. 13.
    Pluim JPW, Maintz JBA, Viergever MA (2000) Image registration by maximization of combined mutual information and gradient information. Med Image Comput Comput-Assist Interv MICCAI 2000 1935:452–461CrossRefGoogle Scholar
  14. 14.
    Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359CrossRefGoogle Scholar
  15. 15.
    Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1–3):203–211CrossRefGoogle Scholar
  16. 16.
    Zhang GY, Wittek A, Joldes GR, Jin X, Miller K (2014) A three-dimensional nonlinear meshfree algorithm for simulating mechanical responses of soft tissue. Eng Anal Bound Elem 42: 60–66CrossRefGoogle Scholar
  17. 17.
    MICCAI (2012) Challenge on multimodal brain tumor segmentation. Accessed 20 Dec 2014
  18. 18.
    Bullitt E, Zeng D, Gerig G, Aylward S, Joshi S, Smith JK, Lin W, Ewend MG (2005) Vessel tortuosity and brain tumor malignancy: a blinded study. Acad Radiol 12:1232–1240PubMedCentralCrossRefPubMedGoogle Scholar
  19. 19.
    Marami B, Sirouspour S, Capson DW (2011) Model-based deformable registration of preoperative 3D to intraoperative low-resolution 3D and 2D sequences of MR images. In: Fichtinger G, Martel A, Peters T (eds) Medical Image Computing Computer-Assisted Intervention—MICCAI 2011, Vol 6891. Springer, Berlin, pp 460–467Google Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  • Jingya Zhang
    • 1
    • 2
  • Jiajun Wang
    • 1
    Email author
  • Xiuying Wang
    • 3
  • Dagan Feng
    • 3
  1. 1.School of Electronic and Information EngineeringSoochow UniversitySuzhouPeople’s Republic of China
  2. 2.Department of PhysicsChangshu Institute of TechnologyChangshuPeople’s Republic of China
  3. 3.Biomedical and Multimedia Information Technology Research Group, School of Information TechnologiesUniversity of SydneySydneyAustralia

Personalised recommendations