Abstract
Image registration is the process of establishing a common geometric reference frame between two or more image data sets possibly taken at different times. In this paper we present a method for computing elastic registration and warping maps based on the Monge–Kantorovich theory of optimal mass transport. This mass transport method has a number of important characteristics. First, it is parameter free. Moreover, it utilizes all of the grayscale data in both images, places the two images on equal footing and is symmetrical: the optimal mapping from image A to image B being the inverse of the optimal mapping from B to A. The method does not require that landmarks be specified, and the minimizer of the distance functional involved is unique; there are no other local minimizers. Finally, optimal transport naturally takes into account changes in density that result from changes in area or volume. Although the optimal transport method is certainly not appropriate for all registration and warping problems, this mass preservation property makes the Monge–Kantorovich approach quite useful for an interesting class of warping problems, as we show in this paper. Our method for finding the registration mapping is based on a partial differential equation approach to the minimization of the L 2 Kantorovich–Wasserstein or “Earth Mover's Distance” under a mass preservation constraint. We show how this approach leads to practical algorithms, and demonstrate our method with a number of examples, including those from the medical field. We also extend this method to take into account changes in intensity, and show that it is well suited for applications such as image morphing.
Similar content being viewed by others
References
Ambrosio, L. 2000. Lecture notes on optimal transport problems. Lectures given at Euro Summer School. Available on http://cvgmt.sns.it/papers/amb00a/.
Angenent, S., Haker, S., and Tannenbaum, A. 2003. Minimizing flows for the Monge-Kantorovich problem. SIAM J. Math. Analysis, 35:61–97.
Angenent, S., Haker, S., Tannenbaum, A., and Kikinis, R. 1999a. On area preserving maps of minimal distortion. In System Theory: Modeling, Analysis, and Control, T. Djaferis and I. Schick (Eds.), Kluwer: Holland, pp. 275–287.
Angenent, S., Haker, S., Tannenbaum, A., and Kikinis, R. 1999b. Laplace-Beltrami operator and brain surface flattening. IEEE Trans. on Medical Imaging, 18:700–711.
Aruliah, D.A., Aschery, U.M., Haberz, E., and Oldenburgx, D. 2001. A method for the forward modelling of 3D electromagnetic quasistatic problems. Mathematical Models and Methods in Applied Sciences, 11:1–21.
Benamou, J.-D. and Brenier,Y. 2000.Acomputational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numerische Mathematik, 84:375–393.
Brenier, Y. 1991. Polar factorization and monotone rearrangement of vector-valued functions. Com. Pure Appl. Math., 64:375–417.
Bro-Nielsen, M. and Gramkow, C. 1996. Fast fluid registration of medical images. In Visualization in Biomedical Imaging, K. Höhne and R. Kikinis (Eds.), Lecture Notes in Computer Science, vol. 1131, Springer-Verlag: New York, pp. 267–276.
Christensen, G.E., Rabbit, R.D., and Miller, M. 1996. Deformable templates using large deformation kinetics. IEEE Trans. on Image Processing, 5:1435–1447.
Christensen, G.E., Rabbit, R.D., and Miller, M. 1993. A deformable neuroanatomy handbook based on viscous fluid mechanics. In 27th Ann. Conf. on Inf. Sciences and Systems, pp. 211–216.
Christensen, G.E. and Johnson, H.J. 2001. Consistent image registration. IEEE Trans. on Medical Imaging, 20:568–582.
Cullen, M. and Purser, R. 1984. An extended Lagrangian theory of semigeostrophic frontogenesis. J. Atmos. Sci., 41:1477–1497.
Dacorogna, B. and Moser, J. 1990. On a partial differential equation involving the Jacobian determinant. Ann. Inst. H. Poincaré Anal.Non Linéaire, 7:1–26.
Fry, D. 1993. Shape recognition using metrics on the space of shapes.Ph.D. Thesis, Harvard University.
Gangbo, W. 1994. An elementary proof of the polar factorization of vector-valued functions. Arch. Rational Mechanics Anal., 128:381–399.
Gangbo, W. and McCann, R. 1996. The geometry of optimal transportation. Acta Math., 177:113–161.
Gangbo, W. and McCann, R. 1999. Shape recognition via Wasserstein distance. Technical Report, School of Mathematics, Georgia Institute of Technology.
Haker, S., Angenent, S., Tannenbaum, A., and Kikinis, R. 2000. Nondistorting flattening maps and the 3D visualization of colon CT images. IEEE Trans. of Medical Imaging.
Haralick, R. and Shapiro, L. 1992. Computer and Robot Vision. Addison-Wesley: New York.
Hinterberger, W. and Scherzer, O. 2001. Models for image interpolation based on the optical flow. Computing, 66:231–247.
Kaijser, T. 1998. Computing the Kantorovich distance for images. Journal of Mathematical Imaging and Vision, 9:173–191.
Kantorovich, L.V. 1948. On a problem of monge. Uspekhi Mat. Nauk., 3:225–226.
Knott, M. and Smith, C. 1984. On the optimal mapping of distributions. J. Optim. Theory, 43:39–49.
Levina, E. and Bickel, P. 2001. The earth mover's distance is the Mallow's distance: Some insights from statistics. In Proceedings IEEE Int. Conf. on Computer Vision, vol. 2, pp. 251–256.
McCann, R. 2001. Polar factorization of maps on Riemannian manifolds. Geom. Funct. Anal., 11:589–608.
McCann, R. 1997. A convexity principle for interacting gases. Adv. Math., 128:153–179.
Miller,M., Christensen, G., Amit,Y., and Grenander,U. 1992. Mathematical textbook of deformable neuroanatomies. Proc. National Academy of Science, 90:11944–11948.
Moser, J. 1965. On the volume elements on a manifold. Trans. Amer. Math. Soc., 120:286–294.
Press, W., Teukolsky, S., Vetterling, W., and Flannery, B. 1992. Numerical Recipes in C: The Art of Scientific Computing, 2nd edn., Cambridge University Press: Cambridge U.K.
Rachev S. and Rüschendorf, L. 1998. Mass Transportation Problems, vols. I and II, Probability and its Applications. Springer: New York.
Rubner, Y. 1999. Perceptual metrics for image database navigation. Ph.D. Thesis, Stanford University.
Rubner, Y., Tomasi, C., and Guibas, J. 1998. The earth mover's distance as a metric for image retrieval. Technical Report STAN-CS-TN-98-86, Department of Computer Science, Stanford University.
Strang, G. 1986. Introduction to Applied Mathematics. Wellesley-Cambridge Press: Wellesley, MA.
Taylor,M. 1996. Partial Differential Equations III. Springer-Verlag: New York.
Thirion, J.-P. 1995. Fast non-rigid matching of non-rigid images. In Medical Robotics and Computer Aided Surgery (MRCAS' 95), Baltimore, p. 4754.
Toga, A. 1999. Brain Warping. Academic Press: San Diego.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Haker, S., Zhu, L., Tannenbaum, A. et al. Optimal Mass Transport for Registration and Warping. International Journal of Computer Vision 60, 225–240 (2004). https://doi.org/10.1023/B:VISI.0000036836.66311.97
Issue Date:
DOI: https://doi.org/10.1023/B:VISI.0000036836.66311.97