Abstract
The fusion of different sources of information has always been a component of medical practice. The complexity of biological phenomena is such that they cannot be explained with a single exploration. The complementary nature of the available exploration techniques (modalities) helps the physician in refining his diagnosis, preparing or performing therapeutic procedures. In this respect it is interesting to note that the development of new medical modalities has not led to the replacement of former ones, and that obviously there is no single modality providing the clinician with all possible sources of information. Before the development of computerised registration tools, data fusion involved pure mental matching of the data sets based on well-known common structures, the matching of the rest of the data being mentally interpolated from this initial step. The parallel emergence of new digital medical imaging devices, communication networks and powerful workstations has made it possible not only to display images but also to transfer and process them. Image processing methods have recently been developed to perform direct, automatic data matching. These methods define the multimodal data fusion topic. They have modified the way multimodal matching is performed as well as the way multimodal information is used; moving from a mental to a computer assisted fusion process. This evolution has led to a more accurate, more visual, more quantitative and therefore more objective fusion process.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Maintz J.A., Viergever M.A., A review of medical image registration, Medical Image Analysis, 1998,2(l):l–36.
Maurer C.R., Fitzpatrick J.M., A Review of Medical Image Registration, Interactive Image-Guided Neurosurgery, Maciunas R.J. Ed., American Association of Neurological Surgeons, USA, 1993, chapter 3:17–44.
Van Den Elsen P.A., Pol E.J.D., Viergever M.A., Medical image matching — a review with classification, IEEE Eng. in Medicine and Biology Magazine, 1993,12(l):26–39.
Hawkes D.J., Algorithms for radiological image registration and their clinical application, J. of Anatomy, 1998,193(3):347–361.
Brown L.G., A survey of image registration techniques, ACM Computing Surveys, 1992, 24(4):325–376.
Duncan R., SPECT Imaging in Focal Epilepsy, SPECT Imaging of the Brain, Kluwer Academic Publishers, 1997, chapter 2:43–68.
Friston K.J., Jezzard P., Turner R., Analysis of functional MRI time-series, Human Brain Mapping, 1994,2(1):153–171.
Worsley K.J., Friston K.J., Analysis of fMRI time-series revisited — again, Neuroimage, 1995, 2:173–181.
Lacey A.J., Thacker N.A., Jackson A., Burton E., Locating motion artefacts in parametric fMRI analysis, Proc. 2nd Inter. Conf. on Medical Image Computing and Computer-Assisted Interventions, 1999, Cambridge, England, Lecture Notes in Computer Science, 1679:524–532.
Behrenbruch C., Marias K., Armitage P., Yam M., Moore N., English R., Brady M., MRI-mammography 2D/3D data fusion for breast pathology assessment, Proc. 3 Inter. Conf. on Medical Image Computing and Computer-Assisted Interventions, 2000, Pittsburgh, USA, Lecture Notes in Computer Science, Springer, 1935:307–316.
Boes, J.L., C.R. Meyer, Multi-variate mutual information for registration, Proc. 2nd Inter. Conf. on Medical Image Computing and Computer-Assisted Interventions, 1999, Cambridge, England, Lecture Notes in Computer Science, 1679:606–612.
Farrell E.J., Gorniak R.J.T., Kramer E.L., Noz M.E., Maguire Jr. G.Q., Reddy D.P., Graphical 3D medical image registration and quantification, J. of Medical Systems, 1997,21(3):155–172.
Kalfas I.H., Kormos D.W., Murphy M.A., Mc Kenzie R.L., Baraett G.H., Bell G.R., Steiner C.P., Trimble M.B., Weisenberger J.P., Application offrameless stereotaxy to pedicle screw fixation of the spine, Journal of Neurosurgery, 1995,83:641–647.
Zana F., Klein J.C., A multimodal registration algorithm of eye fundus images using vessels detection and Hough transform, IEEE Trans, on Medical Imaging, 1999,18(5):419–428.
Lloret D., López A., Serrat J., Villanueva J.J., Creaseness-based computer tomography and magnetic resonance registration: comparison with the mutual information method, J. of Electronic Imaging, 1999,8(3):255–262.
Stefan H., Schneider S., Feistel H., Pawlik G., Schüler P., Abraham-Fuchs K., Schelgel T., Neubauer U., Huk W.J., Ictal and interictal activity in partial epilepsy recorded with multichannel magnetoelectroencephalography: Correlation of Electroencephalography/Electrocorticography, magnetic resonance imaging, single photon emission computed tomography, and positron emission tomography findings, Epilepsia, 1992,33(5):874–887.
Liu A.K., Belliveau J.W., Dale A.M., Spatiotemporal imaging of human brain activity using junctional MRI constrained magnetoencephalogrphy data: Monte Carlo simulations, Proc. Natl. Acad. Sci. USA, 1998,95:8945–8950.
Maurer C.R. Jr., Hill D.L.G., Martin A.J., Liu H., McCue M., Rueckert D., Lloret D., Hall W.A., Maxwell R.E., Hawkes D.J., Truwit C.L., Investigation of intra-operative brain deformation using a L5T interventional MR system: Preliminary results, TREE Trans, on Medical Imaging, 1998, 17(5):817–825.
Comeau R.M., Fenster A., Peters T.M., Intraoperative US imaging in image-guided neurosurgery, Radiographics, 1998,18(4): 1019–1027.
Olivier A., Germano I.M., Cukiert A., Peters T., Frameless stereotaxy for surgery of the epilepsies: Preliminary experience, J. of Neurosurgery, 1994, 81:629–633.
Doyle W.K., Interactive Image-Directed Epilepsy Surgery: Rudimentary virtual reality in neurosurgery, Interactive Technology and the New Paradigm for Healthcare, IOS Press, 1995, chapter 16:91-100.
Tebo S.A., Leopold D.A., Long D.M., Kennedy D.W., Zinreich S.J., An optical 3D digitizer for frameless stereotactic surgery, IEEE Computer Graphics and Applications, 1996,16(1):55–64.
Cuchet E., Knoplioch J., Dormont D., Marsault C., Registration in neurosurgery and neuroradiotherapy applications, J. of Image Guided Surgery, 1995,1:198–207.
Jannin P., Scarabin J.M., Rolland Y., Schwartz D., A real 3d approach for the simulation of neurosurgical stereotactic act, Proc. SPIE Medical Imaging VHI: Image Capture, Formatting, and Display, Newport Beach, USA, 1994,2164:155–166.
Scarabin J.M., Croci S., Jannin P., Romeas R., Maurincomme E., Behague M., Carsin M., A new concept of stereotactic room with multimodal imaging, Proc. 13th Inter. Congress and Exhibition, Computer Assisted Radiology and Surgery, Paris, France, 1999,841–845.
Simon D.A., Herbert M., Kanade T., Techniques for fast and accurate intrasurgical registration, J. of Image Guided Surgery, 1995, l(l):17–29.
Weese J., Penney G.P., Desmedt P., Buzug T.M., Hill D.L.G., Hawkes D.J.H., Voxel-based 2-D/3-D registration of fluoroscopy images and CT scans for image-guided surgery, IEEE Trans, on Information Technology in Biomedicine, 1997, l(4):284–293.
Weese J., Buzug T. M., Lorenz C., Fassnacht C., An approach to 2D/3D registration of a vertebra in 2D X-ray fluoroscopies with 3D CT images, Lecture Notes in Computer Science, 1997, 1205:119–128.
Betting F., Feldmar J., 3D-2D projective registration of anatomical surfaces with their projections, Proc. 14th Information Processing in Medical Imaging Conf., 1995, Ile de Berder, France, 3:275–286.
Evans A.C., Collins D.L., Mills S. R., Brown E. D., Kelly R.L., 3D statistical neuroanatomical models from 305 MRI volumes, Proc. IEEE Nuclear Science Symp. and Medical Imaging Conf., San Francisco, USA, 1993,1813–1817.
Collins D.L., Holmes C.J., Peters T.M., Evans A.C., Automatic 3D model-based neuroanatomical segmentation, Human Brain Mapping, 1995,3(3): 190–208.
Ashburner J., Friston K.J., Multimodal image coregistration and partitioning — a unified framework, Neurolmage, 1997,6(3):209–217.
Mazziotta J.C., Toga A.W., Evans A.C., Fox P., Lancaster J.L., A probabilistic atlas of human brain: Theory and rationales for its development, Neuroimage, 1995,2:89–101.
Friston K.J., Ashburner J., Poline J.B., Frith C.D., Heather J.D., Frackowiak R.S.J., Spatial registration and normalization of images. Human Brain Mapping, 1995,2:165–189.
Guimond A., Roche A., Ayache N., Meunier J., Multimodal brain warping using the demons algorithm and adaptive intensity corrections, INRIA, France, Report N°RR-396,1999.
Barillot C., Fusion de données et imagerie 3D en médecine, Thèse d’habilitauon à dinger des recherches, IRISA, Université de Rennes, France, 1999.
Talairach J., Tournoux P., Co-Planar stereotactic atlas of the human brain, Georg Thieme Verlag, 1988.
Collins D.L., Le Goualher G., Evans A.C., Non-linear cerebral registration with sulcal constraints, Proc. 1st Inter. Conf. on Medical Image Computing and Computer-Assisted Interventions, 1998, Cambridge, USA, Lecture Notes in Computer Science, 1496:974–984.
Van Den Elsen P., Maintz A., Pol E., Viergever M., Automatic registration of CT and MR brain images using correlation of geometrical features, IEEE Trans, on Medical Imaging, 1995, 14(2):384–396.
Subsol G., Thirion J.P., Ayache N., Steps towards automatic building of anatomical atlases, SPIE Proc. 3rd Conf. on Visualization in Biomedical Computing, 1994, Rochester, USA, 2359:435–446.
Faugeras O., Three-Dimensional Computer Vision: A Geometric Viewpoint, MIT Press, 1993.
Bookstein F.L., Principal warps: Thin-plate splines and the decomposition of deformations, IEEE Trans, on Pattern Analysis and Machine Intelligence, 1989,11(6):567–585.
Bookstein F.L., Thin-plate splines and the atlas problem for biomedical images, Proc. 12th Conf. on Information Processing in Medical Imaging, 1991, Wye, England, Lecture Notes in Computer Science, 511:326–342.
Sander S., Leahy R., Surface-based labelling of cortical anatomy using a deformable atlas, IEEE Trans, on Medical Imaging, 1997,16(l):41–54.
Thompson P., Toga A., A surface-based technique for warping three-dimensional images of the brain, IEEE Trans, on Medical Imaging, 1996,15(4):402–417.
Szeliski R., Lavallée S., Matching 3-D anatomical surfaces with non-rigid deformations using octree splines, Inter. J. of Computer Vision, 1996,18(2):176–186.
Collins D.L., Peters T.M., Evans A.C., An automated 3D non-linear deformation procedure for determination of gross morphometric variability in human brain, SPIE Proc. 3 Conf. on Visualization in Biomedical Computing, 1994, Rochester, USA, 2359:180–190.
Hellier P., Barillot C, Memin E„ Perez P., Medical image registration with robust multigrid techniques, Proc. 2nd Inter. Conf. on Medical Image Computing and Computer-Assisted Interventions, 1999, Cambridge, England, Lecture notes in Computer Science, 1679:680–687.
Thirion J.P., Non-rigid matching using demons, Proc. Conf. on Computer Vision and Pattern Recognition, Los Alamitos, USA, IEEE Computer Society Press, 1996,245–251.
Thirion J.P., Image matching as a diffusion process: An analogy with Maxwells demons, Medical Image Analysis, 1998,2(3):243–260.
Bajcsy R., Kovacic S., Multiresolution elastic matching, Computer Vision, Graphics, and Image Processing, 1989,46(1):1–21.
Gee J., Reivicj M., Bajcsy R., Elastically deforming 3D atlas to match anatomical brain images, J. of Computed Assisted Tomography, 1993,17(2):225–236.
Christensen G.E., Rabbitt R.D., Miller M.I., Joshi S.C., Grenander U., Coogan T.A., Van Essen D.C., Topological properties of smooth anatomic maps, Proc. 14th Inter. Conf. on Information Processing in Medical Imaging, 1995, Dordrecht, Netherlands, 3:101–112.
Miller M.I., Christensen G.E., Amit Y.A., Grenander U., Mathematical textbook of deformable neuroanatomies, Medical Sciences, 1993,90:11944–11948.
Borgefors G., A new distance transformation approximating the Euclidean distance, Proc. 8th Inter. Conf. on Pattern Recognition, 1986, Paris, France, 336–338.
Borgefors G., Hierarchical chamfer matching: A parametric edge matching algorithm, IEEE Trans, on Pattern Analysis and Machine Intelligence, 1988,10(6):849–865.
Pelizzari C.A., Chan G. T.Y., Spelhring D.R., Weichselbaum E. E., Chen C.T., Accurate three-dimensional registration of CT, PET and/or MR images of the brain, J. of Computer Assisted Tomography, 1989,13(l):20–26.
Roche A., Malandain G., Ayache N., Pennec X., Multimodal image registration by maximization of the correlation ratio, INRIA, France, Report N°RR-3378,1998.
Roche A., Malandain G., Pennec X., Ayache N., The correlation ratio as a new similarity measure for multimodal image registration, Proc. 1st Inter. Conf. on Medical Image Computing and Computer-Assisted Intervention, 1998, Cambridge, USA, Lecture Notes in Computer Science, 1496:1115–1124.
Woods R.P., Mazziotta J.C., Cherry S.R., MRI-PET registration with automated algorithm, J. of Computer Assisted Tomography, 1993,17(4):536–546.
Maes F., Collignon A., Vandermeulen D., Marchal G., Suetens P., Multimodality image registration by maximization of mutual information, IEEE Trans, on Medical Imaging, 1997, 16(2):187–198.
Wells III W.M., Viola P., Atsumi H., Nakajima S., Kikinis R., Multi-modal volume registration by maximization of mutual information, Medical Image Analysis, 1996,1(1):35–51.
Roche A., Malandain G., Ayache N., Unifying maximum likelihood approaches in medical image registration, INRIA, France, Report N°RR-3741,1999.
De Castro E., Morandi C., Registration of translated and rotated images using finite Fourier transforms, IEEE Trans. Pattern Analysis and Machine Intelligence, 1987,9:700–703.
Pluim J.P.W., Maintz J.B.A., Viergever M.A., Interpolation artefacts in mutual information-based image registration, Computer Vision and Image Understanding, 2000,77(2):211–232.
Maes F., Vendermeulen D., Suetens P., Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximisation of mutual information, Medical Image Analysis, 1999,3(4):373–386.
Black M.J., Rangarajan A., On the unification of line processes, outlier rejection, and robust statistics with applications in early vision, Inter. J. of Computer Vision, 1996,19(1):57–91.
Nikou C., Heitz F., Armspach J.P., Namer I.J., Grucker D., Registration of MR/MR and MR/SPECT brain images by fast stochastic optimization of robust voxel similarity measures, Neuroimage, 1998, 8:30–43.
Press W.H., Teukolsky S.A., Vetterling W.T., Flannery B.P., Numerical Recipes in C, Cambridge University Press, 1992.
Jannin P., Bouliou A., Scarabin J.M., Barillot C., Lubert J., Visual matching between real and virtual images in image guided neurosurgery, SPIE Proc. on Medical Imaging: Image Display, 1997,3031:518–526.
Jannin P., Grova C., Schwartz D., Barillot C., Gibaud B., Visual qualitative comparison between functional neuro-imaging (MEG, fMRI, SPECT), Proc. 13th Computer Assisted Radiology and Surgery Conf., 1999, Paris, France, 238–243.
Viergever M.A., Maintz J.B.A., Stokking R., Integration of Junctional and anatomical brain images, Biophysical Chemistry, 1997,68(l–3):207–219.
Socoloinsky D.A., Wolff L.B., Image fusion for enhanced visualization of brain imaging, SPIE Proc. on Medical Imaging: Image Display, 1999,3658:352–362.
Noordmans H.J., Van der Voort H.T.M., Rutten G.J.M., Viergever M.A., Physically realistic visualization of embedded volume structures for medical image data, SPIE Proc. on Medical Imaging: Image Display, 1999,3658:613–620.
Colin A., Boire J.-Y., MRI-SPECT fusion for the synthesis of high resolution 3d functional brain images: A preliminary study, Computer Methods and Programs in Biomedicine, 1999,60(2): 107–116.
Hawkes D.J., Hill D.L.G., Lehmann E.D., Robinson G.P., Maisey M.N., Colchester A.C.F., Preliminary work on the interpretation of SPECT images with the aid of registered MR images and an MR derived 3D neuro-anatomical atlas, 3D Imaging in Medicine, Springer Verlag, 1990, 241–251.
O’Brien T.J., O’Connor M.K., Mullan B.P., Brinkmann B.H., Hanson D., Jack C.R., So E.L., Subtraction ictal SPECT co-registered to MRI in partial epilepsy: Description and technical validation of the method with phantom and patient studies, Nuclear Medicine Communications, 1998,19:31–45.
Lundervold A., Storvik G., Segmentation of brain parenchyma and cerebrospinal fluid in multispectral magnetic resonance images, IEEE Trans, on Medical Imaging, 1995, 14(2):339–349.
Brinkmann B.H., O’Brien T.J., Aharon S., O’Connor M.K., Mullan B.P., Hanson D.P., Robb R.A., Quantitative and clinical analysis of SPECT image registration for epilepsy studies. The J. of Nuclear Medecine, 1999,40(7):1098–1105.
Le Goualher G., Procyk E., Collins D.L., Venugopal R., Barillot C., Evans A.C., Automated extraction and variability analysis of sulcal neuroanatomy, IEEE Trans, on Medical Imaging, 1999,18(3):206–217.
Friston K.J., Holmes A.P.,Worsley K.J., Poline J.B., Frith C.D., Frackowiak R.S.J., Statistical parametric maps in functional imaging: A general linear approach, Human Brain Mapping, 1995,2:189–210.
Taylor C.A., Draney M.T., Ku J.P., Parker D., Steele B.N., Wang K., Zarins C.K., Predictive medicine: Computational techniques in therapeutic decision-making, Computer aided surgery, 1999,4:231–247.
Sumanaweera T.S., Adler J.R., Napel S., Glover G.H., Characterization of spatial distortion in magnetic resonance imaging and its implications for stereotactic surgery, Neurosurgery, 1994, 35(4):696–704.
Collins D.L., Zijdenbos A.P., Kollokian V., Sled J., Kabani N.J., Holmes C.J., Evans A.C., Design and construction of a realistic digital brain phantom, IEEE Trans, on Medical Imaging, 1998, 17(3):463–468.
West J., Fitzpatrick J.M., Wang M.Y., Dawant B.M., Maurer C.R., Kessler R.M., Maciunas R.J., Barillot C., Lemoine D., Collignon A., Maes F., Suetens P., Vandermeulen D., Van Den Elsen P.A., Napel S., Sumanaweera T.S., Harkness B., Hemler P.F., Hill D.L.G., Hawkes D.J., Studholme C., Maintz J.B.A., Viergever M.A., Malandin G., Pennec X., Noz M.E., Maguire G.Q., Pollack M., Pellizzari C.A., Robb R.A., Hanson D., Woods R., Comparison and evaluation of retrospective intermodality image registration techniques, SPIE Proc. on Medical Imaging: Image Processing, 1996, 2710:332–347.
West J., Fitzpatrick J.M., Wang M.Y., Dawant B.M., Maurer C.R., Kessler R.M., Maciunas R.J., Barillot C., Lemoine D., Collignon A., Maes F., Suetens P., Vandermeulen D., Van Den Elsen P.A., Napel S., Sumanaweera T.S., Harkness B., Hemler P.F., Hill D.L.G., Hawkes D.J., Studholme C., Maintz J.B.A., Viergever M.A., Malandin G., Pennec X., Noz M.E., Maguire G.Q., Pollack M., Pellizzari C.A., Robb R.A., Hanson D., Woods R., Comparison and evaluation of retrospective intermodality brain image registration techniques, J. of Computer Assisted Tomography, 1997, 21(4):554–566,1997.
Van Bemmel J.H., Musen M.A., Handbook of Medical Informatics, Springer Verlag, 1997.
Vieth J., Kober H., Weise E., Daun A., Moeger A., Friedrich, Pongratz H., Functional 3d localization of cerebrovascular accidents by magnetoencephalography (MEG), Neurological Research, 1992,14:132–134.
Schwartz D., Localisation Des Generateurs Intra-Cerebraux de L’activite MEG et EEG: Evaluation de la Precision Spatiale et Temporelle, PhD thesis, Universite de Rennes 1, France, 1997.
Schwartz D., Poiseau E., Lemoine D., Barillot C., Registration of MEG/EEG data with MRI: Methodology and precision issues, Brain Topography, 1996,9(2):101–116.
Schwartz D., Badier J.M., Bihouee P., Bouliou A., Evaluation of a new MEG-EEG spatio-temporal approach using realistic sources, Brain Topography, 1999,11(4):279–289.
Barillot C., Schwartz D.P., LeGoualher G., Gibaud B., Scarabin J.M., Representation of MEG/EEG data in a 3D morphological environment, Computer Assisted Radiology, Proc. of the Inter. Symp. on Computer and Communication Systems for Image Guided Diagnosis and Therapy, 1996, Paris, France, 249–254.
Yoo T.S., Ackerman M.J., Vannier M., Toward a common validation methodology for segmentation and registration algorithms, Proc. 3rd Inter. Conf. on Medical Image Computing and Computer-Assisted Interventions, 2000, Pittsburgh, USA, Lecture Notes in Computer Science, Springer, 1935:422–431.
DICOM Committee, Digital Imaging and Communications in Medecine (DICOM), ACR-NEMA Standard PS3.1-9,1993.
Jannin J., Reig O.J., Seigneuret E., Grova C., Morandi X., Scarabin J.M., A data fusion environment for multimodal and multi-informational neuro-navigation, J. of Computer Aided Surgery, 2000,5(1):1–10.
Satava R.M., Jones S.B., Current and future applications of virtual reality for medicine; Proc. of IEEE, 1998,86(3):484–489.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer Science+Business Media New York
About this chapter
Cite this chapter
Jannin, P., Grova, C., Gibaud, B. (2001). Medical Applications of NDT Data Fusion. In: Gros, X.E. (eds) Applications of NDT Data Fusion. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-1411-4_11
Download citation
DOI: https://doi.org/10.1007/978-1-4615-1411-4_11
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-7923-7412-1
Online ISBN: 978-1-4615-1411-4
eBook Packages: Springer Book Archive