International Journal of Computer Vision

, Volume 126, Issue 1, pp 36–58 | Cite as

Graph-Based Slice-to-Volume Deformable Registration

  • Enzo FerranteEmail author
  • Nikos Paragios


Deformable image registration is a fundamental problem in computer vision and medical image computing. In this paper we investigate the use of graphical models in the context of a particular type of image registration problem, known as slice-to-volume registration. We introduce a scalable, modular and flexible formulation that can accommodate low-rank and high order terms, that simultaneously selects the plane and estimates the in-plane deformation through a single shot optimization approach. The proposed framework is instantiated into different variants seeking either a compromise between computational efficiency (soft plane selection constraints and approximate definition of the data similarity terms through pair-wise components) or exact definition of the data terms and the constraints on the plane selection. Simulated and real-data in the context of ultrasound and magnetic resonance registration (where both framework instantiations as well as different optimization strategies are considered) demonstrate the potentials of our method.


Slice-to-volume registration Graphical models Deformable registration Discrete optimization 



This research was partially supported by European Research Council Starting Grant Diocles (ERC-STG-259112). We thank Mihir Sahasrabudhe for proof-reading the paper, and Puneet Kumar Dokania, Vivien Fecamp and Jorg Kappes for helpful discussions.


  1. Andres, B., Kappes, J. H., Beier, T., Köthe, U., & Hamprecht, F. A. (2012). The lazy flipper: Efficient depth-limited exhaustive search in discrete graphical models. In A. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato & C. Schmid (Eds.), Computer vision (ECCV 2012) (pp. 154–166). Berlin: Springer.Google Scholar
  2. Baudin, P. Y., Goodman, D., Kumar, P., Azzabou, N., Carlier, P. G., Paragios, N., et al. (2013). Discriminative parameter estimation for random walks segmentation. In K. Mori, I. Sakuma, Y. Sato, C. Barillot & N. Navab (Eds.), Medical image computing and computer-assisted intervention (MICCAI 2013) (pp. 219–226). Berlin: Springer.Google Scholar
  3. Besag, J. (1986). On the statistical analysis of dirty pictures. Journal of the Royal Statistical Society Series B (Methodological), 48, 259–302.Google Scholar
  4. Birkfellner, W., Figl, M., Kettenbach, J., Hummel, J., Homolka, P., Schernthaner, R., et al. (2007). Rigid 2D/3D slice-to-volume registration and its application on fluoroscopic CT images. Medical Physics, 34(1), 246. doi: 10.1118/1.2401661.CrossRefGoogle Scholar
  5. Chandler, A. G., Pinder, R. J., Netsch, T., Schnabel, J. A., Hawkes, D. J., Hill, D. L., et al. (2008). Correction of misaligned slices in multi-slice MR cardiac examinations by using slice-to-volume registration. Journal of Cardiovascular Magnetic Resonance, 2008(10), 13.CrossRefGoogle Scholar
  6. Eresen, A., Li, P., & Ji, J. X. (2014). Correlating 2D histological slice with 3D MRI image volume using smart phone as an interactive tool for muscle study. In 2014 36th Annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 6418–6421). IEEE.Google Scholar
  7. Fei, B., Duerk, J. L., & Wilson, D. L. (2002). Automatic 3D registration for interventional MRI-guided treatment of prostate cancer. Computer Aided Surgery, 7(5), 257–267.CrossRefGoogle Scholar
  8. Ferrante, E., Fecamp, V., & Paragios, N. (2015a). Implicit planar and in-plane deformable mapping in medical images through high order graphs. In IEEE international symposium on biomedical imaging: From nano to macro (ISBI).Google Scholar
  9. Ferrante, E., Fecamp, V., & Paragios, N. (2015b). Slice-to-volume deformable registration: Efficient one shot consensus between plane selection and in-plane deformation. International Journal of Computer Assisted Radiology and Surgery (IJCARS), 10(6), 16.Google Scholar
  10. Ferrante, E., & Paragios, N. (2013). Non-rigid 2D–3D medical image registration using Markov random fields. In Medical image computing and computer-assisted intervention (MICCAI 2013) (pp. 163–170). Berlin: Springer.Google Scholar
  11. Ferrante, E., & Paragios, N. (2017). Slice-to-volume medical image registration: A survey. Medical Image Analysis, 39, 101–123.Google Scholar
  12. Gill, S., Abolmaesumi, P., Vikal, S., Mousavi, P., & Fichtinger, G. (2008). Intraoperative prostate tracking with slice-to-volume registration in MRI. In Proceedings of the 20th international conference of the society for medical innovation and technology 2008 (pp. 154–158). Society for Medical Innovation and Technology SMIT.Google Scholar
  13. Glocker, B. (2010). Random fields for image registration. PhD Thesis.Google Scholar
  14. Glocker, B., Komodakis, N., Paragios, N., & Navab, N. (2009). Approximated curvature penalty in non-rigid registration using pairwise MRFs. In G. Bebis, R. Boyle, B. Parvin, D. Koracin, Y. Kuno, J. Wang, et al. (Eds.), Advances in visual computing (pp. 1101–1109). Berlin: Springer.Google Scholar
  15. Glocker, B., Komodakis, N., Tziritas, G., Navab, N., & Paragios, N. (2008). Dense image registration through MRFs and efficient linear programming. Medical Image Analysis, 12(6), 731–741. doi: 10.1016/ Special issue on information processing in medical imaging 2007.CrossRefGoogle Scholar
  16. Glocker, B., Sotiras, A., Komodakis, N., & Paragios, N. (2011). Deformable medical image registration: Setting the state of the art with discrete methods. Annual Review of Biomedical Engineering, 13, 219–244. doi: 10.1146/annurev-bioeng-071910-124649.CrossRefGoogle Scholar
  17. Huang, X., Moore, J., Guiraudon, G., Jones, D. L., Bainbridge, D., Ren, J., et al. (2009). Dynamic 2D ultrasound and 3D ct image registration of the beating heart. IEEE Transactions on Medical Imaging, 28(8), 1179–1189.CrossRefGoogle Scholar
  18. Jiang, S., Xue, H., Counsell, S., Anjari, M., Allsop, J., Rutherford, M., et al. (2009). Diffusion tensor imaging (DTI) of the brain in moving subjects: Application to in-utero fetal and ex-utero studies. Magnetic Resonance in Medicine, 62(3), 645–655.CrossRefGoogle Scholar
  19. Kappes, J. H., Andres, B., Hamprecht, F. A., Schnörr, C., Nowozin, S., Batra, D., et al. (2013). A comparative study of modern inference techniques for discrete energy minimization problems. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2013 (pp. 1328–1335).Google Scholar
  20. Kim, B., Boes, J. L., Bland, P. H., Chenevert, T. L., & Meyer, C. R. (1999). Motion correction in fMRI via registration of individual slices into an anatomical volume. Magnetic Resonance in Medicine, 41(5), 964–972.Google Scholar
  21. Kohli, P., & Rother, C. (2012). Higher-order models in computer vision. Chapter 1, Image processing and analysis with graphs. Boca Raton: CRC Press.zbMATHGoogle Scholar
  22. Komodakis, N., Paragios, N., & Tziritas, G. (2011). MRF energy minimization and beyond via dual decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3), 531–552.Google Scholar
  23. Komodakis, N., Tziritas, G., & Paragios, N. (2007). Fast, approximately optimal solutions for single and dynamic MRFs. In IEEE conference on computer vision and pattern recognition, 2007 (CVPR’07) (pp. 1–8). IEEE.
  24. Komodakis, N., Xiang, B., & Paragios, N. (2015). A framework for efficient structured max-margin learning of high-order MRF models. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 37(7), 1425–1441.Google Scholar
  25. Kschischang, F. R., Frey, B. J., & Loeliger, H. A. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 47(2), 498–519.MathSciNetCrossRefzbMATHGoogle Scholar
  26. Kwon, D., Lee, K. J., Yun, I. D., & Lee, S. U. (2008). Nonrigid image registration using dynamic higher-order MRF model. In D. Forsyth, P. Torr & A. Zisserman (Eds.), Computer Vision (ECCV 2008) (pp. 373–386). Berlin: Springer.Google Scholar
  27. Leung, K. Y. E., van Stralen, M., Nemes, A., Voormolen, M. M., van Burken, G., Geleijnse, M. L., et al. (2008). Sparse registration for three-dimensional stress echocardiography. IEEE Transactions on Medical Imaging, 27(11), 1568–1579.Google Scholar
  28. Liao, R., Zhang, L., Sun, Y., Miao, S., & Chefd’Hotel, C. (2013). A review of recent advances in registration techniques applied to minimally invasive therapy. IEEE Transactions on Multimedia, 15(5), 983–1000.CrossRefGoogle Scholar
  29. Markelj, P., Tomaževič, D., Likar, B., & Pernuš, F. (2012). A review of 3D/2D registration methods for image-guided interventions. Medical Image Analysis, 16(3), 642–661.Google Scholar
  30. Mercier, L., Del Maestro, R. F., Petrecca, K., Araujo, D., Haegelen, C., & Collins, D. L. (2012). Online database of clinical MR and ultrasound images of brain tumors. Medical Physics, 39, 3253.CrossRefGoogle Scholar
  31. Murphy, K. P., Weiss, Y., & Jordan, M. I. (1999). Loopy belief propagation for approximate inference: An empirical study. In Proceedings of the fifteenth conference on uncertainty in artificial intelligence (UAI’99).Google Scholar
  32. Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal, 7(4), 308–313. doi: 10.1093/comjnl/7.4.308.MathSciNetCrossRefzbMATHGoogle Scholar
  33. Olesch, J., Beuthien, B., Heldmann, S., Papenberg, N., & Fischer, B. (2011). Fast intra-operative non-linear registration of 3D-ct to tracked, selected 2D-ultrasound slices. In SPIE medical imaging (pp. 79642R–79642R). International Society for Optics and Photonics.Google Scholar
  34. Osechinskiy, S., & Kruggel, F. (2011). Slice-to-volume nonrigid registration of histological sections to MR images of the human brain. Anatomy Research International, 2011, 1–17. doi: 10.1155/2011/287860.CrossRefGoogle Scholar
  35. Paragios, N., Ferrante, E., Glocker, B., Komodakis, N., Parisot, S., & Zacharaki, E. I. (2016). (Hyper)-graphical models in biomedical image analysis. Medical Image Analysis, 33, 102–106.Google Scholar
  36. Paragios, N., Komodakis, N. (2014). Discrete visual perception. In 2014 22nd International conference on pattern recognition (ICPR) (pp. 18–25). IEEE.Google Scholar
  37. Porchetto, R., Stramana, F., Paragios, N., & Ferrante, E. (2016). Rigid slice-to-volume medical image registration through Markov random fields. BAMBI Workshop, MICCAI 2016.Google Scholar
  38. Ramalingam, S., Kohli, P., Alahari, K., & Torr, P. H. (2008). Exact inference in multi-label crfs with higher order cliques. In IEEE conference on computer vision and pattern recognition, 2008 (CVPR 2008) (pp. 1–8). IEEE.Google Scholar
  39. Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L., Leach, M. O., & Hawkes, D. J. (1999). Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging, 18(8), 712–721.CrossRefGoogle Scholar
  40. San José Estépar, R., Westin, C., & Vosburgh, K. (2009). Towards real time 2D to 3D registration for ultrasound-guided endoscopic and laparoscopic procedures. International Journal of Computer Assisted Radiology and Surgery, 4(6), 549–560.CrossRefGoogle Scholar
  41. Seshamani, S., Fogtmann, M., Cheng, X., Thomason, M., Gatenby, C., & Studholme, C. (2013). Cascaded slice to volume registration for moving fetal fMRI. In 2013 IEEE 10th international symposium on biomedical imaging (ISBI) (pp. 796–799). IEEE.Google Scholar
  42. Shekhovtsov, A., Kovtun, I., & Hlaváč, V. (2008). Efficient MRF deformation model for non-rigid image matching. Computer Vision and Image Understanding, 112(1), 91–99.CrossRefGoogle Scholar
  43. Sotiras, A., Davatzikos, C., & Paragios, N. (2013). Deformable medical image registration: A survey. IEEE Transactions on Medical Imaging, 32(7), 1153–1190.CrossRefGoogle Scholar
  44. Sotiras, A., Ou, Y., Glocker, B., Davatzikos, C., & Paragios, N. (2010). Simultaneous geometric–iconic registration. In T. Jiang, N. Navab, J. P. W. Pluim & M. A. Viergever (Eds.), Medical image computing and computer-assisted intervention (MICCAI 2010) (pp. 676–683). Berlin: Springer.Google Scholar
  45. Wang, C., Komodakis, N., & Paragios, N. (2013). Markov random field modeling, inference & learning in computer vision & image understanding: A survey. Computer Vision and Image Understanding, 117(11), 1610–1627.CrossRefGoogle Scholar
  46. Xu, H., Lasso, A., Fedorov, A., Tuncali, K., Tempany, C., & Fichtinger, G. (2014). Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy. International Journal of Computer Assisted Radiology and Surgery, 10(5), 1–10.Google Scholar
  47. Zikic, D., Glocker, B., Kutter, O., Groher, M., Komodakis, N., Kamen, A., et al. (2010). Linear intensity-based image registration by Markov random fields and discrete optimization. Medical Image Analysis, 14(4), 550–562.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Center for Visual Computing, CentraleSupelec, INRIAUniversite Paris-SaclayParisFrance
  2. 2.Biomedical Image Analysis (BioMedIA) Group, Department of ComputingImperial College LondonLondonUK
  3. 3.TheraPanaceaParisFrance

Personalised recommendations