Deformable Registration of Multi-modal Microscopic Images Using a Pyramidal Interactive Registration-Learning Methodology

  • Tingying Peng
  • Mehmet Yigitsoy
  • Abouzar Eslami
  • Christine Bayer
  • Nassir Navab
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8545)


Co-registration of multi-modal microscopic images can integrate benefits of each modality, yet major challenges come from inherent difference between staining, distortions of specimens and various artefacts. In this paper, we propose a new interactive registration-learning method to register functional fluorescence (IF) and structural histology (HE) images in a pyramidal fashion. We synthesize HE image from the multi-channel IF image using a supervised machine learning technique and hence reduce the multi-modality registration problem into a mono-modality one, in which case the normalised cross correlation is used as the similarity measure. Unlike conventional applications of supervised learning, our classifier is not trained by ‘ground-truth’ (perfectly-registered) training dataset, as they are not available. Instead, we use a relatively noisy training dataset (affinely-registered) as an initialization and rely on the robustness of machine learning to the outliers and label updates via pyramidal deformable registration to gain better learning and predictions. In this sense, the proposed methodology has potential to be adapted in other learning problems as the manual labelling is usually imprecise and very difficult in the case of heterogeneous tissues.


Microscopy multimodality deformable registration noisy robust supervised learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pitkeathly, W.T.E., Poulter, N.S., Claridge, E., Rappoport, J.Z.: Auto-align - multi-modality fluorescence microscopy image co-registration. Traffic 13, 204–217 (2012)CrossRefGoogle Scholar
  2. 2.
    Wells, W.M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R.: Multi-modal volume registration by maximization of mutual information. Med. Image Anal. 1, 35–51 (1996)CrossRefGoogle Scholar
  3. 3.
    Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22, 986–1004 (2003)CrossRefGoogle Scholar
  4. 4.
    Zhuang, X., Arridge, S., Hawkes, D.J., Ourselin, S.: A Non-rigid Registration Framework Using Spatially Encoded Mutual Information and Free-Form Deformations. IEEE Trans. Med. Imaging 30, 1819–1828 (2011)CrossRefGoogle Scholar
  5. 5.
    Wachinger, C., Navab, N.: Entropy and Laplacian images: structural representations for multi-modal registration. Med. Image Anal. 16, 1–17 (2012)CrossRefGoogle Scholar
  6. 6.
    Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, S.M., Schnabel, J.A.: MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Image Anal. 16, 1423–1435 (2012)CrossRefGoogle Scholar
  7. 7.
    Wein, W., Brunke, S., Khamene, A., Callstrom, M.R., Navab, N.: Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Med. Image Anal. 12, 577–585 (2008)CrossRefGoogle Scholar
  8. 8.
    Burgos, N., et al.: Attenuation correction synthesis for hybrid PET-MR scanners. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 147–154. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  9. 9.
    Ou, Y., Sotiras, A., Paragios, N., Davatzikos, C.: DRAMMS: Deformable registration via attribute matching and mutual-saliency weighting. Med. Image Anal. 15, 622–639 (2011)CrossRefGoogle Scholar
  10. 10.
    Tang, L.Y.W., Hamarneh, G.: Random Walks with efficient search and contextually adapted image similarity for deformable registration. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 43–50. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  11. 11.
    Warfield, S., Kaus, M., Jolesz, F.A., Kikinis, R.: Adaptive, template moderated, spatially varying statistical classification. Med. Image Anal. 4, 43–55 (2000)CrossRefGoogle Scholar
  12. 12.
    Studholme, C., Hill, D.L.G., Hawkes, D.J.: An overlap invariant entropy measure of 3D medical image alignment. Pattern Recognit. 32, 71–86 (1999)CrossRefGoogle Scholar
  13. 13.
    Ostu, N.: A Threshold Selection Method from Gray-Level Histograms. IEEE Trans. Syst. Man Cybern. 9, 62–66 (1979)CrossRefGoogle Scholar
  14. 14.
    Ruifrok, A.C., Johnston, D.A.: Quantification of histochemical staining by color deconvolution. Anal. Quant. Cytol. Histol. 23, 291–299 (2001)Google Scholar
  15. 15.
    Freund, Y.: A more robust boosting algorithm. arXiv:0905.2138v1 (2009)Google Scholar
  16. 16.
    Allwein, E.L.: Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers. J. Mach. Learn. Res. 1, 113–141 (2000)MathSciNetGoogle Scholar
  17. 17.
    Avants, B.B., Duda, J.T., Zhang, H., Gee, J.C.: Multivariate normalization with symmetric diffeomorphisms for multivariate studies. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part I. LNCS, vol. 4791, pp. 359–366. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Song, Y., Treanor, D., Bulpitt, A., Wijayathunga, N., Roberts, N., Wilcox, R., Magee, D.: Unsupervised Content Classification Based Non-rigid Registration of Differently Stained Histology Images. IEEE Trans. Biomed. Eng. (2013)Google Scholar
  19. 19.
    Michel, F., Paragios, N.: Image transport regression using mixture of experts and discrete Markov Random Fields. In: 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1229–1232. IEEE (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Tingying Peng
    • 1
  • Mehmet Yigitsoy
    • 1
  • Abouzar Eslami
    • 1
  • Christine Bayer
    • 2
  • Nassir Navab
    • 1
  1. 1.Computer Aided Medical Procedures (CAMP)Technical University of MunichGermany
  2. 2.Department of Radiation OncologyTechnical University of MunichGermany

Personalised recommendations