Abstract
Imaging with multiple modalities or multiple channels is becoming increasingly important for our modern society. A key tool for understanding and early diagnosis of cancer and dementia is PET-MR, a combined positron emission tomography and magnetic resonance imaging scanner which can simultaneously acquire functional and anatomical data. Similarly, in remote sensing, while hyperspectral sensors may allow to characterize and distinguish materials, digital cameras offer high spatial resolution to delineate objects. In both of these examples, the imaging modalities can be considered individually or jointly. In this chapter we discuss mathematical approaches which allow combining information from several imaging modalities so that multi-modality imaging can be more than just the sum of its components.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
van Aarle, W., Palenstijn, W.J., Cant, J., Janssens, E., Bleichrodt, F., Dabravolski, A., De Beenhouwer, J., Joost Batenburg, K., Sijbers, J.: Fast and flexible X-ray tomography using the ASTRA toolbox. Optics Express 24(22), 25129 (2016). https://doi.org/10.1364/OE.24.025129
van Aarle, W., Palenstijn, W.J., De Beenhouwer, J., Altantzis, T., Bals, S., Batenburg, K.J., Sijbers, J.: The ASTRA Toolbox: A platform for advanced algorithm development in electron tomography. Ultramicroscopy 157, 35–47 (2015). https://doi.org/10.1016/j.ultramic.2015.05.002
Adler, J., Kohr, H., Öktem, O.: Operator Discretization Library (ODL) (2017). https://doi.org/10.5281/zenodo.249479
Ametova, E., Fardell, G., Jørgensen, J.S., Lionheart, W.R.B., Papoutsellis, E., Pasca, E., Sykes, D., Turner, M., Warr, R., Withers, P.J.: Core Imaging Library (CIL) (2019). https://www.ccpi.ac. uk/cil
Arridge, S.R., Burger, M., Ehrhardt, M.J.: Preface to special issue on joint reconstruction and multi-modality/multi-spectral imaging. Inverse Prob. 36, 020302 (2020)
Arridge, S.R., Kolehmainen, V., Schweiger, M.J.: Reconstruction and regularisation in optical tomography. In: Censor, A., Jiang, Y., Louis, M. (eds.) Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy (IMRT). Scuola Normale Superiore (2008)
Arridge, S.R., Simmons, A.: Multi-spectral probabilistic diffusion using Bayesian classification. In: ter Haar Romeny, B.M., Florack, L., Koenderink, J.J., Viergever M.A. (eds.) Scale-Space Theories in Computer Vision, pp. 224–235. Springer, Berlin (1997). https://doi.org/10.1007/3-540-63167-4_53
Baete, K., Nuyts, J., Van Paesschen, W., Suetens, P., Dupont, P.: Anatomical-based FDG-PET reconstruction for the detection of hypo-metabolic regions in epilepsy. IEEE Trans. Med. Imaging 23(4), 510–519 (2004). https://doi.org/10.1109/TMI.2004.825623
Bai, B., Li, Q., Leahy, R.M.: Magnetic resonance-guided positron emission tomography image reconstruction. Semin. Nucl. Med. 43, 30–44 (2013). https://doi.org/10.1053/j.semnuclmed.2012.08.006
Ballester, C., Caselles, V., Igual, L., Verdera, J., Rougé, B.: A variational model for P+XS image fusion. Int. J. Comput. Vis. 69(1), 43–58 (2006). https://doi.org/10.1007/s11263-006-6852-x
Bathke, C., Kluth, T., Maass, P.: Improved image reconstruction in magnetic particle imaging using structural a priori information. Int. J. Magn. Part. Imaging 3(1) (2017)
Bauschke, H.H., Combettes, P.L.: Convex analysis and monotone operator theory in Hilbert spaces (2011). https://doi.org/10.1007/978-1-4419-9467-7
Benning, M., Burger, M.: Modern regularization methods for inverse problems. Acta Numerica 27, 1–111 (2018). https://doi.org/10.1017/S0962492918000016
Bilgic, B., Goyal, V.K., Adalsteinsson, E.: Multi-contrast reconstruction with Bayesian compressed sensing. Magn. Reson. Med. 66(6), 1601–1615 (2011). https://doi.org/10.1002/mrm.22956
Blomgren, P., Chan, T.F.: Color TV: Total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7(3), 304–309 (1998). https://doi.org/10.1109/83.661180
Bousse, A., Pedemonte, S., Kazantsev, D., Ourselin, S., Arridge, S.R., Hutton, B.F.: Weighted MRI-based Bowsher priors for SPECT brain image reconstruction. In: IEEE Nuclear Science Symposium and Medical Imaging Conference, pp. 3519–3522 (2010)
Bousse, A., Pedemonte, S., Thomas, B.A., Erlandsson, K., Ourselin, S., Arridge, S.R., Hutton, B.F.: Markov random field and Gaussian mixture for segmented MRI-based partial volume correction in PET. Phys. Med. Biol. 57(20), 6681–6705 (2012). https://doi.org/10.1088/0031-9155/57/20/6681
Bowsher, J.E., Johnson, V.E., Turkington, T.G., Jaszczak, R.J., Floyd, C.E., Coleman, R.E.: Bayesian reconstruction and use of anatomical a priori information for emission tomography. IEEE Trans. Med. Imaging 15(5), 673–686 (1996). https://doi.org/10.1109/42.538945
Bowsher, J.E., Yuan, H., Hedlund, L.W., Turkington, T.G., Akabani, G., Badea, A., Kurylo, W.C., Wheeler, C.T., Cofer, G.P., Dewhirst, M.W., Johnson, G.A.: Utilizing MRI information to estimate F18-FDG distributions in rat flank tumors. In: IEEE Nuclear Science Symposium and Medical Imaging Conference, pp. 2488–2492 (2004). https://doi.org/10.1109/NSSMIC.2004.1462760
Bredies, K., Dong, Y., Hintermüller, M.: Spatially dependent regularization parameter selection in total generalized variation models for image restoration. Int. J. Comput. Math. 1–15 (2012). https://doi.org/10.1080/00207160.2012.700400
Bredies, K., Holler, M.: Regularization of linear inverse problems with total generalized variation. J. Inverse Ill-Posed Prob. 22(6), 871–913 (2014). https://doi.org/10.1515/jip-2013-0068
Bredies, K., Holler, M.: A TGV-based framework for variational image decompression, zooming, and reconstruction. Part II: Numerics. SIAM J. Imag. Sci. 8(4), 2851–2886 (2015). https://doi.org/10.1137/15M1023877
Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imag. Sci. 3(3), 492–526 (2010). https://doi.org/10.1137/090769521
Bredies, K., Lorenz, D.A.: Mathematical Image Processing, 1 edn. Birkhäuser Basel (2018). https://doi.org/10.1007/978-3-030-01458-2
Bresson, X., Chan, T.F.: Fast dual minimization of the vectorial total variation norm and applications to color image processing. Inverse Prob. Imaging 2(4), 455–484 (2008). https://doi.org/10.3934/ipi.2008.2.455
Bungert, L., Coomes, D.A., Ehrhardt, M.J., Rasch, J., Reisenhofer, R., Schönlieb, C.B.: Blind image fusion for hyperspectral imaging with the directional total variation. Inverse Prob. 34(4), 044003 (2018). https://doi.org/10.1088/1361-6420/aaaf63
Bungert, L., Ehrhardt, M.J.: Robust image reconstruction with misaligned structural information (2020). http://arxiv.org/abs/2004.00589
Bungert, L., Ehrhardt, M.J., Reisenhofer, R.: Robust blind image fusion for misaligned hyperspectral imaging data. In: Proceedings in Applied Mathematics & Mechanics, vol. 18, p. e201800033 (2018). https://doi.org/10.1002/pamm.201800033
Burger, M., Osher, S.: A guide to the TV zoo. In: Level Set and PDE Based Reconstruction Methods in Imaging, Lecture Notes in Mathematics, vol. 2090, pp. 1–70. Springer (2013). https://doi.org/10.1007/978-3-319-01712-9
Caselles, V., Coll, B., Morel, J.M.: Geometry and color in natural images. J. Math. Imaging Vision 16(Section 2), 89–105 (2002). https://doi.org/10.1023/A:1013943314097
Chambolle, A., Ehrhardt, M.J., Richtárik, P., Schönlieb, C.B.: Stochastic primal-dual hybrid gradient algorithm with arbitrary sampling and imaging applications. SIAM J. Optim. 28(4), 2783–2808 (2018). https://doi.org/10.1007/s10851-010-0251-1
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40(1), 120–145 (2011). https://doi.org/10.1007/s10851-010-0251-1
Chambolle, A., Pock, T.: An introduction to continuous optimization for imaging. Acta Numerica 25, 161–319 (2016). https://doi.org/10.1017/S096249291600009X
Chan, C., Fulton, R., Feng, D.D., Cai, W., Meikle, S.: An anatomically based regionally adaptive prior for MAP reconstruction in emission tomography. In: IEEE Nuclear Science Symposium and Medical Imaging Conference, pp. 4137–4141 (2007). https://doi.org/10.1109/NSSMIC.2007.4437032
Chan, C., Fulton, R., Feng, D.D., Meikle, S.: Regularized image reconstruction with an anatomically adaptive prior for positron emission tomography. Phys. Med. Biol. 54(24), 7379–400 (2009). https://doi.org/10.1088/0031-9155/54/24/009
Chen, C., Li, Y., Huang, J.: Calibrationless parallel MRI with joint total variation regularization. In: Medical Image Computing and Computer-Assisted Intervention, pp. 106–114 (2013). https://doi.org/10.1007/978-3-642-40760-4_14
Cheng-Liao, J., Qi, J.: PET image reconstruction with anatomical edge guided level set prior. Phys. Med. Biol. 56, 6899–6918 (2011). https://doi.org/10.1088/0031-9155/56/21/009
Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. Springer Optim. Appl. 49, 185–212 (2011). https://doi.org/10.1007/978-1-4419-9569-8_10
Comtat, C., Kinahan, P.E., Fessler, J.A., Beyer, T., Townsend, D.W., Defrise, M., Michel, C.J.: Clinically feasible reconstruction of 3D whole-body PET/CT data using blurred anatomical labels. Phys. Med. Biol. 47(1), 1–20 (2002)
Davies, M., Puy, G., Vandergheynst, P., Wiaux, Y.: A compressed sensing framework for magnetic resonance fingerprinting. SIAM J. Imag. Sci. 7(4), 2623–2656 (2013). https://doi.org/10.1137/130947246
Deidda, D., Karakatsanis, N.A., Robson, P.M., Tsai, Y.J., Efthimiou, N., Thielemans, K., Fayad, Z.A., Aykroyd, R.G., Tsoumpas, C.: Hybrid PET-MR list-mode kernelized expectation maximization reconstruction. Inverse Prob. 35(4) (2019). https://doi.org/10.1088/1361-6420/ab013f
Deligiannis, N., Mota, J.F., Cornelis, B., Rodrigues, M.R., Daubechies, I.: Multi-modal dictionary learning for image separation with application in art investigation. IEEE Trans. Image Process. 26(2), 751–764 (2017). https://doi.org/10.1109/TIP.2016.2623484
Delso, G., Furst, S., Jakoby, B., Ladebeck, R., Ganter, C., Nekolla, S.G., Schwaiger, M., Ziegler, S.I., Fürst, S., Jakoby, B., Ladebeck, R., Ganter, C., Nekolla, S.G., Schwaiger, M., Ziegler, S.I.: Performance measurements of the Siemens mMR integrated whole-body PET/MR scanner. J. Nucl. Med. 52(12), 1914–22 (2011). https://doi.org/10.2967/jnumed.111.092726
Di, Z.W., Leyffer, S., Wild, S.M.: Optimization-based approach for joint X-Ray fluorescence and transmission tomographic inversion. SIAM J. Imag. Sci. 9(1), 1–23 (2016)
Dong, G., Hintermüller, M., Papafitsoros, K.: Quantitative magnetic resonance imaging: From fingerprinting to integrated physics-based models. SIAM J. Imag. Sci. 12(2), 927–971 (2019). https://doi.org/10.1137/18M1222211
Dong, Y., Hintermüller, M., Rincon-Camacho, M.M.: Automated regularization parameter selection in multi-scale total variation models for image restoration. J. Math. Imaging Vision 40(1), 82–104 (2011). https://doi.org/10.1007/s10851-010-0248-9
Duran, J., Buades, A., Coll, B., Sbert, C., Blanchet, G.: A survey of pansharpening methods with a new band-decoupled variational model. ISPRS J. Photogramm. Remote Sens. 125, 78–105 (2017). https://doi.org/10.1016/j.isprsjprs.2016.12.013
Ehrhardt, M.J.: Joint reconstruction for multi-modality imaging with common structure. Ph.d. thesis, University College London (2015)
Ehrhardt, M.J., Arridge, S.R.: Vector-valued image processing by parallel level sets. IEEE Trans. Image Process. 23(1), 9–18 (2014). https://doi.org/10.1109/TIP.2013.2277775
Ehrhardt, M.J., Betcke, M.M.: Multi-contrast MRI reconstruction with structure-guided total variation. SIAM J. Imag. Sci. 9(3), 1084–1106 (2016). https://doi.org/10.1137/15M1047325
Ehrhardt, M.J., Markiewicz, P.J., Liljeroth, M., Barnes, A., Kolehmainen, V., Duncan, J., Pizarro, L., Atkinson, D., Hutton, B.F., Ourselin, S., Thielemans, K., Arridge, S.R.: PET reconstruction with an anatomical MRI prior using parallel level sets. IEEE Trans. Med. Imaging 35(9), 2189–2199 (2016). https://doi.org/10.1109/TMI.2016.2549601
Ehrhardt, M.J., Markiewicz, P.J., Schönlieb, C.B.: Faster PET reconstruction with non-smooth priors by randomization and preconditioning. Phys. Med. Biol. 64(22), 225019 (2019). https://doi.org/10.1088/1361-6560/ab3d07
Ehrhardt, M.J., Thielemans, K., Pizarro, L., Atkinson, D., Ourselin, S., Hutton, B.F., Arridge, S.R.: Joint reconstruction of PET-MRI by exploiting structural similarity. Inverse Prob. 31(1), 015001 (2015). https://doi.org/10.1088/0266-5611/31/1/015001
Ehrhardt, M.J., Thielemans, K., Pizarro, L., Markiewicz, P.J., Atkinson, D., Ourselin, S., Hutton, B.F., Arridge, S.R.: Joint reconstruction of PET-MRI by parallel level sets. In: IEEE Nuclear Science Symposium and Medical Imaging Conference (2014). https://doi.org/10.1109/NSSMIC.2014.7430895
Elbau, P., Mindrinos, L., Scherzer, O.: Quantitative reconstructions in multi-modal photoacoustic and optical coherence tomography imaging. Inverse Prob. 34(1) (2018). https://doi.org/10.1088/1361-6420/aa9ae7
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Mathematics and Its Applications. Springer (1996)
Esser, E., Zhang, X., Chan, T.F.: A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J. Imag. Sci. 3(4), 1015–1046 (2010). https://doi.org/10.1137/09076934X
Estellers, V., Soatto, S., Bresson, X.: Adaptive regularization with the structure tensor. IEEE Trans. Image Process. 24(6), 1777–1790 (2015). https://doi.org/10.1109/TIP.2015.2409562
Estellers, V., Thiran, J., Bresson, X.: Enhanced compressed sensing recovery with level set normals. IEEE Trans. Image Process. 22(7), 2611–2626 (2013). https://doi.org/10.1109/TIP.2013.2253484
Fang, F., Li, F., Shen, C., Zhang, G.: A variational approach for pan-sharpening. IEEE Trans. Image Process. 22(7), 2822–2834 (2013). https://doi.org/10.1109/TIP.2013.2258355
Fessler, J.A., Elbakri, I., Sukovic, P., Clinthorne, N.H.: Maximum-likelihood dual-energy tomographic image reconstruction. In: SPIE: Medical Imaging, vol. 4684, pp. 1–25 (2002). https://doi.org/doi:10.1117/12.467189
Foygel Barber, R., Sidky, E.Y., Gilat Schmidt, T., Pan, X.: An algorithm for constrained one-step inversion of spectral CT data. Phys. Med. Biol. 61(10), 3784–3818 (2016). https://doi.org/10.1088/0031-9155/61/10/3784
Gallardo, L.A., Meju, M.A.: Characterization of heterogeneous near-surface materials by joint 2D inversion of DC resistivity and seismic data. Geophys. Res. Lett. 30(13), 1658 (2003). https://doi.org/10.1029/2003GL017370
Gallardo, L.A., Meju, M.A.: Joint two-dimensional DC resistivity and seismic travel time inversion with cross-gradients constraints. J. Geophys. Res. 109(B3), 1–11 (2004). https://doi.org/10.1029/2003JB002716
Gallardo, L.A., Meju, M.A.: Structure-coupled multiphysics imaging in geophysical sciences. Rev. Geophys. 49, 1–19 (2011). https://doi.org/10.1029/2010RG000330.1.INTRODUCTION
Golbabaee, M., Chen, Z., Wiaux, Y., Davies, M.: CoverBLIP: accelerated and scalable iterative matched-filtering for magnetic resonance fingerprint reconstruction. Inverse Prob. 36(1), 015003 (2020). https://doi.org/10.1088/1361-6420/ab4c9a
Goldluecke, B., Strekalovskiy, E., Cremers, D.: The natural vectorial total variation which arises from geometric measure theory. SIAM J. Imag. Sci. 5(2), 537–563 (2012). https://doi.org/10.1137/110823766
Haber, E., Holtzman-Gazit, M.: Model fusion and joint inversion. Surv. Geophys. (34), 675–695 (2013). https://doi.org/10.1007/s10712-013-9232-4
Haber, E., Modersitzki, J.: Intensity gradient based registration and fusion of multi-modal images. In: Medical Image Computing and Computer-Assisted Intervention, vol. 46, pp. 726–733. Springer, Berlin/Heidelberg (2006). https://doi.org/10.1160/ME9046
Haber, E., Oldenburg, D.W.: Joint inversion: A structural approach. Inverse Prob. 13, 63–77 (1997). https://doi.org/10.1088/0266-5611/13/1/006
Heismann, B., Schmidt, B., Flohr, T.: Spectral Computed Tomography. SPIE Press (2012)
Hintermüller, M., Rincon-Camacho, M.M.: Expected absolute value estimators for a spatially adapted regularization parameter choice rule in L1-TV-based image restoration. Inverse Prob. 26(8), 085005 (2010). https://doi.org/10.1088/0266-5611/26/8/085005
Holt, K.M.: Total nuclear variation and jacobian extensions of total variation for vector fields. IEEE Trans. Image Process. 23(9), 3975–3989 (2014). https://doi.org/10.1109/TIP.2014.2332397
Huang, J., Chen, C., Axel, L.: Fast Multi-contrast MRI reconstruction. Magn. Reson. Imaging 32(10), 1344–52 (2014). https://doi.org/10.1016/j.mri.2014.08.025
Huber, R., Haberfehlner, G., Holler, M., Bredies, K.: Total generalized variation regularization for multi-modal electron tomography. Nanoscale 1–38 (2019). https://doi.org/10.1039/c8nr09058k
Ito, K., Jin, B.: Inverse Problems – Tikhonov Theory and Algorithms. World Scientific Publishing (2014). https://doi.org/10.1142/9120
Kaipio, J.P., Kolehmainen, V., Vauhkonen, M., Somersalo, E.: Inverse problems with structural prior information. Inverse Prob. 15(3), 713–729 (1999). https://doi.org/10.1088/0266-5611/15/3/306
Kazantsev, D., Arridge, S.R., Pedemonte, S., Bousse, A., Erlandsson, K., Hutton, B.F., Ourselin, S.: An anatomically driven anisotropic diffusion filtering method for 3D SPECT reconstruction. Phys. Med. Biol. 57(12), 3793–3810 (2012). https://doi.org/10.1088/0031-9155/57/12/3793
Kazantsev, D., Jørgensen, J.S., Andersen, M.S., Lionheart, W.R., Lee, P.D., Withers, P.J.: Joint image reconstruction method with correlative multi-channel prior for x-ray spectral computed tomography. Inverse Prob. 34(6) (2018). https://doi.org/10.1088/1361-6420/aaba86
Kazantsev, D., Lionheart, W.R.B., Withers, P.J., Lee, P.D.: Multimodal image reconstruction using supplementary structural information in total variation regularization. Sens. Imaging 15(1), 97 (2014). https://doi.org/10.1007/s11220-014-0097-5
Kimmel, R., Malladi, R., Sochen, N.: Images as embedded maps and minimal surfaces: movies, color, texture, and volumetric medical images. Int. J. Comput. Vis. 39(2), 111–129 (2000). https://doi.org/10.1023/A:1008171026419
Knoll, F., Holler, M., Koesters, T., Otazo, R., Bredies, K., Sodickson, D.K.: Joint MR-PET reconstruction using a multi-channel image regularizer. IEEE Trans. Med. Imaging 36(1) (2016). https://doi.org/10.1109/TMI.2016.2564989
Knoll, F., Koesters, T., Otazo, R., Boada, F., Sodickson, D.K.: Simultaneous MR-PET reconstruction using multi sensor compressed sensing and joint sparsity. In: International Society for Magnetic Resonance in Medicine, vol. 22 (2014)
Kolehmainen, V., Ehrhardt, M.J., Arridge, S.R.: Incorporating structural prior information and sparsity into EIT using parallel level sets. Inverse Prob. Imaging 13(2), 285–307 (2019). https://doi.org/10.3934/ipi.2019015
Leahy, R.M., Yan, X.: Incorporation of anatomical MR data for improved functional imaging with PET. In: Information Processing in Medical Imaging, pp. 105–120. Springer (1991). https://doi.org/10.1007/BFb0033746
Lenzen, F., Berger, J.: Solution-driven adaptive total variation regularization. In: SSVM, pp. 203–215 (2015). https://doi.org/10.1007/978-3-642-24785-9
Loncan, L., De Almeida, L.B., Bioucas-Dias, J.M., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., Liao, W., Licciardi, G.A., Simoes, M., Tourneret, J.Y., Veganzones, M.A., Vivone, G., Wei, Q., Yokoya, N.: Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3(3), 27–46 (2015). https://doi.org/10.1109/MGRS.2015.2440094
Long, Y., Fessler, J.A.: Multi-material decomposition using statistical image reconstruction for spectral CT. IEEE Trans. Med. Imaging 33(8), 1614–1626 (2014). https://doi.org/10.1109/TMI.2014.2320284
Ma, D., Gulani, V., Seiberlich, N., Liu, K., Sunshine, J.L., Duerk, J.L., Griswold, M.A.: Magnetic resonance fingerprinting. Nature 495(7440), 187–92 (2013). https://doi.org/10.1038/nature11971
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16(2), 187–98 (1997). https://doi.org/10.1109/42.563664
Mehranian, A., Belzunce, M., Prieto, C., Hammers, A., Reader, A.J.: Synergistic PET and SENSE MR image reconstruction using joint sparsity regularization. IEEE Trans. Med. Imaging 37(1), 20–34 (2018). https://doi.org/10.1109/TMI.2017.2691044
Mehranian, A., Belzunce, M.A., Niccolini, F., Politis, M., Prieto, C., Turkheimer, F., Hammers, A., Reader, A.J.: PET image reconstruction using multi-parametric anato-functional. Phys. Med. Biol. (2017). https://doi.org/10.1042/BJ20101136%3E
Meju, M.A., Mackie, R.L., Miorelli, F., Saleh, A.S., Miller, R.V.: Structurally-tailored 3D anisotropic CSEM resistivity inversion with cross-gradients criterion and simultaneous model calibration. Geophysics 84(6), 1–62 (2019). https://doi.org/10.1190/geo2018-0639.1
Möller, M., Brinkmann, E.M., Burger, M., Seybold, T.: Color Bregman TV. SIAM J. Imag. Sci. 7(4), 2771–2806 (2014). https://doi.org/10.1137/130943388
Möller, M., Wittman, T., Bertozzi, A.L., Burger, M.: A variational approach for sharpening high dimensional images. SIAM J. Imag. Sci. 5(1), 150–178 (2012). https://doi.org/10.1137/100810356
Nuyts, J.: The use of mutual information and joint entropy for anatomical priors in emission tomography. In: IEEE Nuclear Science Symposium and Medical Imaging Conference, pp. 4149–4154. IEEE (2007). https://doi.org/10.1109/NSSMIC.2007.4437034
Obert, A.J., Gutberlet, M., Kern, A.L., Kaireit, T.F., Grimm, R., Wacker, F., Vogel-Claussen, J.: 1H-guided reconstruction of 19F gas MRI in COPD patients. Magn. Reson. Med. 1–11 (2020). https://doi.org/10.1002/mrm.28209
Parikh, N., Boyd, S.P.: Proximal algorithms. Found Trends Optim 1(3), 123–231 (2014). https://doi.org/10.1561/2400000003
Pedemonte, S., Bousse, A., Hutton, B.F., Arridge, S.R., Ourselin, S.: Probabilistic graphical model of SPECT/MRI. In: Machine Learning in Medical Imaging, pp. 167–174 (2011). https://doi.org/10.1007/978-3-642-24319-6_21
Pluim, J.P.W., Maintz, J.B.A., Viergever, M.A.: Image registration by maximization of combined mutual information and gradient information. IEEE Trans. Med. Imaging 19(8), 809–14 (2000). https://doi.org/10.1109/42.876307
Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1762–1769 (2011). https://doi.org/10.1109/ICCV.2011.6126441
Rangarajan, A., Hsiao, I.T., Gindi, G.: A Bayesian joint mixture framework for the integration of anatomical information in functional image reconstruction. J. Math. Imaging Vision 12(3), 199–217 (2000). https://doi.org/10.1023/A:1008314015446
Rasch, J., Brinkmann, E.M., Burger, M.: Joint reconstruction via coupled bregman iterations with applications to PET-MR imaging. Inverse Prob. 34(1), 014001 (2018a). https://doi.org/10.1088/1361-6420/aa9425
Rasch, J., Kolehmainen, V., Nivajarvi, R., Kettunen, M., Gröhn, O., Burger, M., Brinkmann, E.M.: Dynamic MRI reconstruction from undersampled data with an anatomical prescan. Inverse Prob. 34(7) (2018b). https://doi.org/10.1088/1361-6420/aac3af
Rigie, D., La Riviere, P.: Joint reconstruction of multi-channel, spectral CT data via constrained total nuclear variation minimization. Phys. Med. Biol. 60, 1741–1762 (2015). https://doi.org/10.1088/0031-9155/60/4/1741
Rigie, D.S., Sanchez, A.A., La Riviére, P.J.: Assessment of vectorial total variation penalties on realistic dual-energy CT data. Phys. Med. Biol. 62(8), 3284–3298 (2017). https://doi.org/10.1088/1361-6560/aa6392
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenom. 60(1), 259–268 (1992). https://doi.org/10.1016/0167-2789(92)90242-F
Sapiro, G., Ringach, D.L.: Anisotropic diffusion of multivalued images with applications to color filtering. IEEE Trans. Image Process. 5(11), 1582–1586 (1996). https://doi.org/10.1109/83.541429
Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M., Lenzen, F.: Variational Methods in Imaging, vol. 167 . Springer, New York/London (2008)
Schmitt, U., Louis, A.K.: Efficient algorithms for the regularization of dynamic inverse problems: I. Theory. Inverse Problems 18(3), 645–658 (2002). https://doi.org/10.1088/0266-5611/18/3/308
Schmitt, U., Louis, A.K., Wolters, C., Vaukhonen, M.: Efficient algorithms for the regularization of dynamic inverse problems: II. Applications. Inverse Prob. 18(3), 659–676 (2002). https://doi.org/10.1088/0266-5611/18/3/308
Schramm, G., Holler, M., Rezaei, A., Vunckx, K., Knoll, F., Bredies, K., Boada, F., Nuyts, J.: Evaluation of parallel level sets and Bowsher’s method as segmentation-free anatomical priors for time-of-flight PET reconstruction. IEEE Trans. Med. Imaging 62(2), 590–603 (2017). https://doi.org/10.1109/TMI.2017.2767940
Schuster, T., Hahn, B., Burger, M.: Dynamic inverse problems: Modelling – Regularization – numerics. Inverse Prob. 34(4) (2018). https://doi.org/10.1088/1361-6420/aab0f5
Sochen, N., Kimmel, R., Malladi, R.: A general framework for low level vision. IEEE Trans. Image Process. 7(3), 310–318 (1998). https://doi.org/10.1109/83.661181
Sodickson, D.K., Feng, L., Knoll, F., Cloos, M., Ben-Eliezer, N., Axel, L., Chandarana, H., Block, K.T., Otazo, R.: The rapid imaging renaissance: Sparser samples, denser dimensions, and glimmerings of a grand unified tomography. In: Proceedings of SPIE, vol. 9417, pp. 94170G1–9417014 (2015). https://doi.org/10.1117/12.2085033
Somayajula, S., Panagiotou, C., Rangarajan, A., Li, Q., Arridge, S.R., Leahy, R.M.: PET image reconstruction using information theoretic anatomical priors. IEEE Trans. Med. Imaging 30(3), 537–549 (2011). https://doi.org/10.1109/TMI.2010.2076827
Song, P., Deng, X., Mota, J.F.C., Deligiannis, N., Dragotti, P.L., Rodrigues, M.: Multimodal image super-resolution via joint sparse representations induced by coupled dictionaries. IEEE Trans. Comput. Imaging 1–1 (2019). https://doi.org/10.1109/tci.2019.2916502
Song, P., Weizman, L., Mota, J.F., Eldar, Y.C., Rodrigues, M.R.: Coupled dictionary learning for multi-contrast MRI reconstruction. In: International Conference on Image Processing, 2, pp. 2880–2884 (2018). https://doi.org/10.1109/ICIP.2018.8451341
Tang, J., Rahmim, A.: Bayesian PET image reconstruction incorporating anato-functional joint entropy. Phys. Med. Biol. 54(23), 7063–75 (2009). https://doi.org/10.1088/0031-9155/54/23/002
Tang, J., Rahmim, A.: Anatomy assisted PET image reconstruction incorporating multi-resolution joint entropy. Phys. Med. Biol. 60(1), 31–48 (2015). https://doi.org/10.1088/0031-9155/60/1/31
Tang, S., Fernandez-Granda, C., Lannuzel, S., Bernstein, B., Lattanzi, R., Cloos, M., Knoll, F., Asslander, J.: Multicompartment magnetic resonance fingerprinting. Inverse Prob. 34(9) (2018). https://doi.org/10.1088/1361-6420/aad1c3
Tsai, Y.J., Member, S., Bousse, A., Ahn, S., Charles, W., Arridge, S., Hutton, B.F., Member, S., Thielemans, K.: Algorithms for solving misalignment issues in penalized PET/CT reconstruction using anatomical priors. In: IEEE Nuclear Science Symposium and Medical Imaging Conference Proceedings (NSS/MIC). IEEE (2018)
Tschumperlé, D., Deriche, R.: Vector-valued image regularization with PDEs: A common framework for different applications. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 506–517 (2005). https://doi.org/10.1109/TPAMI.2005.87
Vunckx, K., Atre, A., Baete, K., Reilhac, A., Deroose, C.M., Van Laere, K., Nuyts, J.: Evaluation of three MRI-based anatomical priors for quantitative PET brain imaging. IEEE Trans. Med. Imaging 31(3), 599–612 (2012). https://doi.org/10.1109/TMI.2011.2173766
Wang, G., Zhang, J., Gao, H., Weir, V., Yu, H., Cong, W., Xu, X., Shen, H., Bennett, J., Furth, M., Wang, Y., Vannier, M.: Towards omni-tomography – grand fusion of multiple modalities for simultaneous interior tomography. PloS one 7(6), e39700 (2012). https://doi.org/10.1371/journal.pone.0039700
Wells III, W.M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R.: Multi-modal volume registration by maximization of mutual information. Med. Image Anal. 1(1), 35–51 (1996)
Xi, Y., Zhao, J., Bennett, J., Stacy, M., Sinusas, A., Wang, G.: Simultaneous CT-MRI reconstruction for constrained imaging geometries using structural coupling and compressive sensing. IEEE Trans. Biomed. Eng. (2015). https://doi.org/10.1109/TBME.2015.2487779
Xiang, L., Chen, Y., Chang, W., Zhan, Y., Lin, W., Wang, Q., Shen, D.: Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Trans. Biomed. Eng. 66(7), 2105–2114 (2019). https://doi.org/10.1109/TBME.2018.2883958
Yokoya, N., Grohnfeldt, C., Chanussot, J.: Hyperspectral and multispectral data fusion: A comparative review of the recent literature. IEEE Geosci. Remote Sens. Mag. 5(2), 29–56 (2017). https://doi.org/10.1109/MGRS.2016.2637824
Zhang, Y., Zhang, X.: PET-MRI joint reconstruction with common edge weighted total variation regularization. Inverse Prob. 34(6), 065006 (2018). https://doi.org/10.1088/1361-6420/aabce9
Acknowledgements
The author acknowledges support from the EPSRC grant EP/S026045/1 and the Faraday Institution EP/T007745/1. Moreover, the author is grateful to all his collaborators which indirectly contributed to this chapter over the last couple of years.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this entry
Cite this entry
Ehrhardt, M.J. (2023). Multi-modality Imaging with Structure-Promoting Regularizers. In: Chen, K., Schönlieb, CB., Tai, XC., Younes, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-98661-2_58
Download citation
DOI: https://doi.org/10.1007/978-3-030-98661-2_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98660-5
Online ISBN: 978-3-030-98661-2
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering