Skip to main content
SpringerLink
Log in

Multimodal Medical Image Fusion in Cardiovascular Applications

  • Chapter
  • First Online:
Medical Imaging Technology

Part of the book series: Lecture Notes in Bioengineering ((LNBE))

Abstract

Recent publications in the field of medical image fusion point out the value of multi-modality in diagnosis, pre-surgical planning and surgical intervention. The integration of multiple data sources including medical images from different devices or sensors strongly increases reliability and information content. Successfully fused multi-modal data should not contain any artefacts, not remove any relevant information from the original data and minimize redundancy. Image and data fusion aims at providing supplementary clinical information that is not apparent in the individual images alone. Image and data fusion finds many different applications in the fields of remote sensing, military, biometrics, machine vision and medical imaging. The scientific community has established three levels of fusion rules, namely pixel, feature and decision level. Depending on the application, processing technique or available data each level has its importance and proven significance in medical data processing. Each level provides a set of rules that can be applied. The selection of the fusion operator has a strong impact on the quality of the result. It becomes apparent that the selection of level and technique must vary according to the information that needs to be extracted for a certain application. Each technique has its advantages and disadvantages which have to be carefully evaluated. Based on the availability of multimodal devices, such as ultrasound (US), magnetic resonance imaging (MRI) and computed tomography (CT), different images and data of the same object are obtained. The multiple images, the variety of fusion levels and rules lead to an uncountable number of possible combinations. This makes it very difficult for the user to select the most beneficial solution without losing valuable time and resources. Recent research results show great potential is the development of holistic systems that allow the application of different levels in order to take advantage of the value of each individual processing step to optimize the resulting information. This chapter explains the state of the art in cardiovascular medical image fusion. Multimodal image exploitation in the context of cardiovascular plaque detection is selected as application to illustrate the great potential of a multimodal approach comprising diagnosis as well as pre-surgical planning and the intra-operative process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. World Health Organization (2013) Global status report on noncommunicable diseases 2010. World Health Organization, Geneva, p 2011

    Google Scholar 

  2. Mathers CD, Loncar D (2006) Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med 3(11):2011–2030. doi:10.1371/journal.pmed.0030442

    Article  Google Scholar 

  3. American Heart Association (2014) Why cholesterol matters. Atherosclerosis 2014

    Google Scholar 

  4. Finn AV, Nakano M, Narula J, Kolodgie FD, Virmani R (2010) Concept of vulnerable/unstable plaque. Arterioscler Thromb Vasc Biol 30(7):1282–1292. doi:10.1161/atvbaha.108.179739

    Article  Google Scholar 

  5. Li X, Li J, Jing J, Ma T, Liang S, Zhang J, Mohar D, Raney A, Mahon S, Brenner M, Patel P, Shung KK, Zhou Q, Chen Z (2014) Integrated IVUS-OCT imaging for atherosclerotic plaque characterization. IEEE J Sel Top Quantum Electron 20(2):196–203

    Google Scholar 

  6. Corti R, Fuster V (2011) Imaging of atherosclerosis: Magnetic resonance imaging. Eur Heart J 32(14):1709–1719

    Article  Google Scholar 

  7. Darwish SM (2013) Multi-level fuzzy contourlet-based image fusion for medical applications. IET Image Proc 7(7):694–700

    Article  Google Scholar 

  8. Patil U, Mudengudi U (2011) Image fusion using hierarchical PCA. In: Proceedings of international conference on image information processing (ICIIP), 3–5 Nov 2011, pp 1–6. doi:10.1109/ICIIP.2011.6108966

  9. James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fusion 19(0):4–19. doi:10.1016/j.inffus.2013.12.002

  10. Mishra A, Rakshit S (2008) Fusion of noisy multi-sensor imagery. Defence Sci J 58(1):136–146

    Article  Google Scholar 

  11. Chellamuthu C (2014) Medical image fusion based on hybrid intelligence. Appl Soft Comput 20:83–94

    Article  Google Scholar 

  12. Zhao P, Liu G, Hu C, Huang H, He B (2013) Medical image fusion algorithm based on the laplace-PCA. In: Proceedings of Chinese intelligent automation conference, 2013. Springer, pp 787–794

    Google Scholar 

  13. Alfano B, Ciampi M, Pietro G (2007) A wavelet-based algorithm for multimodal medical image fusion. In: Falcidieno B, Spagnuolo M, Avrithis Y, Kompatsiaris I, Buitelaar P (eds) Semantic multimedia, vol 4816. Lecture Notes in Computer Science. Springer, Berlin, pp 117–120. doi:10.1007/978-3-540-77051-0_13

  14. Ali F, El-Dokany I, Saad A, Abd El-Samie F (2010) A curvelet transform approach for the fusion of MR and CT images. J Mod Opt 57(4):273–286

    Article  MATH  Google Scholar 

  15. Flotats A, Knuuti J, Gutberlet M, Marcassa C, Bengel F, Kaufmann P, Rees M, Hesse B (2011) Hybrid cardiac imaging: SPECT/CT and PET/CT. A joint position statement by the European Association of Nuclear Medicine (EANM), the European Society of Cardiac Radiology (ESCR) and the European Council of Nuclear Cardiology (ECNC). Eur J Nucl Med Mol Imaging 38(1):201–212. doi:10.1007/s00259-010-1586-y

    Article  Google Scholar 

  16. Gaemperli O, Bengel FM, Kaufmann PA (2011) Cardiac hybrid imaging. Eur Heart J 32(17):2100–2108. doi:10.1093/eurheartj/ehr057

    Article  Google Scholar 

  17. Saraste A, Knuuti J (2012) Cardiac PET, CT, and MR: what are the advantages of hybrid imaging? Curr Cardiol Rep 14(1):24–31. doi:10.1007/s11886-011-0231-0

    Article  Google Scholar 

  18. Bourantas CV, Garcia-Garcia HM, Naka KK, Sakellarios A, Athanasiou L, Fotiadis DI, Michalis LK, Serruys PW (2013) hybrid intravascular imaging: current applications and prospective potential in the study of coronary atherosclerosis. J Am Coll Cardiol 61 (13):1369–1378. doi:10.1016/j.jacc.2012.10.057

  19. Van Der Hoeven BL, Schalij MJ, Delgado V (2012) Multimodality imaging in interventional cardiology. Nat Rev Cardiol 9(6):333–346

    Article  Google Scholar 

  20. Pohl C, Van Genderen JL (1998) Review article multisensor image fusion in remote sensing: Soncepts, methods and applications. Int J Remote Sens 19(5):823–854. doi:10.1080/014311698215748

    Article  Google Scholar 

  21. Jameel A, Ghafoor A, Riaz MM (2014) Improved guided image fusion for magnetic resonance and computed tomography imaging. Sci World J 2014

    Google Scholar 

  22. Kor S, Tiwary U (2004) Feature level fusion of multimodal medical images in lifting wavelet transform domain. In: Proceedings of the 26th annual international conference of the IEEE engineering in medicine and biology society (IEMBS’04), pp 1479–1482. doi:10.1109/IEMBS.2004.1403455

  23. Hall DL, Llinas J (1997) An introduction to multisensor data fusion. Proc IEEE 85(1):6–23. doi:10.1109/5.554205

    Article  Google Scholar 

  24. Al-Wassai FA, Kalyankar N, Al-Zaky AA (2011) Multisensor images fusion based on feature-level. arXiv preprint arXiv:11084098

    Google Scholar 

  25. Paul M, Smith L, Monaghan M (2014) Echocardiography. Medicine. doi:10.1016/j.mpmed.2014.05.015

    Google Scholar 

  26. Blum A, Nahir M (2013) Future non-invasive imaging to detect vascular plaque instability and subclinical non-obstructive atherosclerosis. J Geriatr Cardiol 10(2):178–185

    Google Scholar 

  27. Tarkin J, Joshi F, Rudd JF (2013) Advances in molecular imaging: plaque imaging. Curr Cardiovasc Imaging Rep 6(4):358–368. doi:10.1007/s12410-013-9207-3

    Article  Google Scholar 

  28. Wang P, Ecabert O, Chen T, Wels M, Rieber J, Ostermeier M, Comaniciu D (2013) Image-based co-registration of angiography and intravascular ultrasound images. IEEE Trans Med Imaging 32(12):2338–2349

    Article  Google Scholar 

  29. (2013) Artherosclerosis: clinical perspectives through imaging. doi:10.1007/978-1-4471-4288-1

  30. Leber AW, Becker A, Knez A, von Ziegler F, Sirol M, Nikolaou K, Ohnesorge B, Fayad ZA, Becker CR, Reiser M, Steinbeck G, Boekstegers P (2006) Accuracy of 64-slice computed tomography to classify and quantify plaque volumes in the proximal coronary system: a comparative study using intravascular ultrasound. J Am Coll Cardiol 47(3):672–677. doi:10.1016/j.jacc.2005.10.058

  31. Nissen SE, Gurley JC, Grines CL, Booth DC, McClure R, Berk M, Fischer C, DeMaria AN (1991) Intravascular ultrasound assessment of lumen size and wall morphology in normal subjects and patients with coronary artery disease. Circulation 84(3):1087–1099. doi:10.1161/01.cir.84.3.1087

    Article  Google Scholar 

  32. Nicholls SJ, Tuzcu EM, Sipahi I, Schoenhagen P, Nissen SE (2006) Intravascular ultrasound in cardiovascular medicine. Circulation 114(4):e55–e59

    Article  Google Scholar 

  33. Wahle A, Prause GPM, Von Birgelen C, Erbel R, Sonka M (1999) Fusion of angiography and intravascular ultrasound in vivo: establishing the absolute 3-D frame orientation. IEEE Trans Biomed Eng 46(10):1176–1180

    Article  Google Scholar 

  34. Pasterkamp G, Falk E, Woutman H, Borst C (2000) Techniques characterizing the coronary atherosclerotic plaque: influence on clinical decision making? J Am Coll Cardiol 36(1):13–21

    Article  Google Scholar 

  35. Patwari P, Weissman NJ, Boppart SA, Jesser C, Stamper D, Fujimoto JG, Brezinski ME (2000) Assessment of coronary plaque with optical coherence tomography and high-frequency ultrasound. Am J Cardiol 85(5):641–644

    Article  Google Scholar 

  36. Suter MJ, Nadkarni SK, Weisz G, Tanaka A, Jaffer FA, Bouma BE, Tearney GJ (2011) Intravascular optical imaging technology for investigating the coronary artery. JACC: Cardiovasc Imaging 4(9):1022–1039. doi:10.1016/j.jcmg.2011.03.020

    Google Scholar 

  37. Gallino A, Stuber M, Crea F, Falk E, Corti R, Lekakis J, Schwitter J, Camici P, Gaemperli O, Di Valentino M, Prior J, Garcia-Garcia HM, Vlachopoulos C, Cosentino F, Windecker S, Pedrazzini G, Conti R, Mach F, De Caterina R, Libby P (2012) In vivo imaging of atherosclerosis. Atherosclerosis 224(1):25–36. doi:10.1016/j.atherosclerosis.2012.04.007

  38. Beller GA (2010) Recent advances and future trends in multimodality cardiac imaging. Heart Lung Circ 19(3):193–209. doi:10.1016/j.hlc.2009.11.003

  39. Beller GA (2010) Recent advances and future trends in multimodality cardiac imaging. Heart Lung Circ 19(3):193–209

    Article  Google Scholar 

  40. Matthäus C, Cicchi R, Meyer T, Lattermann A, Schmitt M, Romeike BFM, Krafft C, Dietzek B, Brehm BR, Pavone FS, Popp J (2014) Multimodal nonlinear imaging of atherosclerotic plaques differentiation of triglyceride and cholesterol deposits. J Innov Opt Health Sci (2)

    Google Scholar 

  41. Rogers I, Tawakol A (2011) Imaging of coronary inflammation with FDG-PET: feasibility and clinical hurdles. Curr Cardiol Rep 13(2):138–144. doi:10.1007/s11886-011-0168-3

    Article  Google Scholar 

  42. Saito H, Kuroda S, Hirata K, Magota K, Shiga T, Tamaki N, Yoshida D, Terae S, Nakayama N, Houkin K (2013) Validity of dual MRI and F-FDG PET imaging in predicting vulnerable and inflamed carotid plaque. Cerebrovasc Dis 35(4):370–377

    Article  Google Scholar 

  43. Baohua Z, Xiaoqi L, Weitao J (2013) A multi-focus image fusion algorithm based on an improved dual-channel PCNN in NSCT domain. Int J Light Electron Opt 124(20):4104–4109. doi:10.1016/j.ijleo.2012.12.032

  44. Mehta S, Marakarkandy B (2013) CT and MRI image fusion using curvelet transform. J Inf Knowl Res Electron Commun Eng 2(2):848–852

    Google Scholar 

  45. Pibarot P, Larose É, Dumesnil J (2013) Imaging of valvular heart disease. Can J Cardiol 29(3):337–349. doi:10.1016/j.cjca.2012.11.006

  46. Roujol S, Basha TA, Tan A, Khanna V, Chan RH, Moghari MH, Rayatzadeh H, Shaw JL, Josephson ME, Nezafat R (2013) Improved multimodality data fusion of late gadolinium enhancement MRI to left ventricular voltage maps in ventricular tachycardia ablation. IEEE Trans Biomed Eng 60(5):1308–1317

    Article  Google Scholar 

  47. Gorpas D, Fatakdawala H, Bec J, Ma D, Yankelevich DR, Bishop JW, Qi J, Marcu L (2014) Bi-modal imaging of atherosclerotic plaques: automated method for co-registration between fluorescence lifetime imaging and intravascular ultrasound data. In: Proceedings of SPIE progress in biomedical optics and imaging

    Google Scholar 

  48. Boogers MJ, Broersen A, Van Velzen JE, De Graaf FR, El-Naggar HM, Kitslaar PH, Dijkstra J, Delgado V, Boersma E, De Roos A, Schuijf JD, Schalij MJ, Reiber JHC, Bax JJ, Jukema JW (2012) Automated quantification of coronary plaque with computed tomography: Comparison with intravascular ultrasound using a dedicated registration algorithm for fusion-based quantification. Eur Heart J 33(8):1007–1016

    Article  Google Scholar 

  49. Godbout B, De Guise JA, Soulez G, Cloutier G (2005) 3D elastic registration of vessel structures from IVUS data on biplane angiography. Acad Radiol 12(1):10–16

    Article  Google Scholar 

  50. Marquering HA, Dijkstra J, Besnehard QJA, Duthé JPM, Schuijf JD, Bax JJ (2008) Reiber JHC Coronary CT angiography—IVUS image fusion for quantitative plaque and stenosis analyses

    Google Scholar 

  51. Pohl C, van Genderen J (2013) Remote sensing image fusion: an update in the context of Digital Earth. Int J Digital Earth 7(2):158–172. doi:10.1080/17538947.2013.869266

    Article  Google Scholar 

  52. Mitchell HB (2010) Image fusion: theories, techniques and applications. Springer, Heidelberg

    Google Scholar 

  53. Saroglu E, Bektas F, Musaoglu N, Goksel C (2004) Fusion of multisensor remote sensing data: assessing the quality of resulting images. Int Arch Photogram Rem Sens Spatial Inform Sci 35:575–579

    Google Scholar 

  54. Shutao L, Xudong K, Jianwen H (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875. doi:10.1109/TIP.2013.2244222

    Article  Google Scholar 

  55. Li C, Yang X, Chu B, Lu W, Pang L (2010) A new image fusion quality assessment method based on contourlet and SSIM. In: Proceedings of 3rd IEEE international conference on computer science and information technology (ICCSIT), pp 246–249

    Google Scholar 

  56. Deshmukh M, Bhosale U (2010) Image fusion and image quality assessment of fused images. Int J Image Process (IJIP) 4(5):484

    Google Scholar 

  57. Galande A, Patil R (2013) The art of medical image fusion: a survey. In: Proceedings of IEEE international conference on advances in computing, communications and informatics (ICACCI), pp 400–405

    Google Scholar 

  58. Li L, Jiang W, Li J, Yuchi M, Ding M, Zhang X A (2013) New assessment method for image fusion quality. In: SPIE medical imaging international society for optics and photonics, pp 86731G-86731G-86736

    Google Scholar 

  59. Yang Y, Tong S, Huang S, Lin P (2014) Log-gabor energy based multimodal medical image fusion in NSCT domain. Comput Math Methods Med 2014

    Google Scholar 

  60. Liu Z, Yin H, Chai Y, Yang SX (2014) A novel approach for multimodal medical image fusion. Expert Syst Appl 41(16):7425–7435. doi:10.1016/j.eswa.2014.05.043

    Article  Google Scholar 

  61. Bedi S, Agarwal MJ, Agarwal P (2013) Image fusion techniques and quality assessment parameters for clinical diagnosis: a review. Int J Adv Res Comput Commun Eng 2(2):2319–5940

    Google Scholar 

  62. Zhang B (2014) Medical fusion image quality assessment based on SSIM. In: Zhong S (ed) Proceedings of the 2012 international conference on cybernetics and informatics, vol 163. Lecture Notes in Electrical Engineering. Springer, New York, pp 1905–1911. doi:10.1007/978-1-4614-3872-4_244

  63. Zhou W, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84. doi:10.1109/97.995823

    Article  Google Scholar 

  64. Piella G, Heijmans H (2003) A new quality metric for image fusion. In: Proceedings of international conference on image processing (ICIP 2003), vol 172, pp III-173-176

    Google Scholar 

  65. Dammavalam SR, Maddala S, Prasad M (2013) Quality assessment of pixel-level imagefusion using fuzzy logic. arXiv preprint arXiv:13111223

    Google Scholar 

  66. Han S, Li H, Gu H (2008) The study on image fusion for high spatial resolution remote sensing images. Int Arch Photogram Rem Sens Spatial Inform Sci 37:1159–1163

    Google Scholar 

  67. Yakhdani MF, Azizi A (2010) Quality assessment of image fusion techniques for multisensor high resolution satellite images (case study: IRS-P5 and IRS-P6 satellite images)

    Google Scholar 

  68. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: Proceedings of the 20th international conference on pattern recognition (ICPR). IEEE, pp 2366–2369

    Google Scholar 

  69. Aja-Fernández S, Alberola-López C (2006) On the estimation of the coefficient of variation for anisotropic diffusion speckle filtering. IEEE Trans Image Process 15(9):2694–2701

    Article  Google Scholar 

  70. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Geoscience and Digital Earth Centre (Geo-Dec), Research Institute for Sustainability and Environment (RISE), Universiti Teknologi Malaysia (UTM), Block T06 / FGHT, 81310, Johor Bahru, Johor, Malaysia

    Christine Pohl

  2. IJN-UTM Cardiovascular Engineering Centre, Faculty of Biosciences and Medical Engineering, Universiti Teknologi Malaysia, 81310, Johor Bahru, Johor, Malaysia

    Nor Nisha Nadhira Nazirun & Nur’Aqilah Hamzah

  3. Institut Jantung Negara/National Heart Institute (IJN), 50400, Kuala Lumpur, Malaysia

    Syahidah Syed Tamin

Authors
  1. Christine Pohl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nor Nisha Nadhira Nazirun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nur’Aqilah Hamzah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Syahidah Syed Tamin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christine Pohl .

Editor information

Editors and Affiliations

  1. Biomedical Engineering, University Malaya, Kuala Lumpur, Malaysia

    Khin Wee Lai

  2. IJN-UTM Cardiovascular Engineering Centr, University Technology Malaysia, Johor Bahru, Malaysia

    Dyah Ekashanti Octorina Dewi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Science+Business Media Singapore

About this chapter

Cite this chapter

Pohl, C., Nazirun, N.N.N., Hamzah, N., Tamin, S.S. (2015). Multimodal Medical Image Fusion in Cardiovascular Applications. In: Lai, K., Octorina Dewi, D. (eds) Medical Imaging Technology. Lecture Notes in Bioengineering. Springer, Singapore. https://doi.org/10.1007/978-981-287-540-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-981-287-540-2_4

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-287-539-6

  • Online ISBN: 978-981-287-540-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation