Skip to main content
SpringerLink
Log in

Variational Methods in Image Matching and Motion Extraction

  • Chapter
  • First Online:
Level Set and PDE Based Reconstruction Methods in Imaging

Part of the book series: Lecture Notes in Mathematics ((LNMCIME,volume 2090))

  • 2121 Accesses

Abstract

In this chapter we are concerned with variational methods in image analysis. Special attention is paid on free discontinuity approaches of Mumford Shah type and their application in segmentation, matching and motion analysis. We study combined approaches, where one simultaneously relaxes a functional with respect to multiple unknowns. Examples are the simultaneous extraction of edges in two different images for joint image segmentation and image registration or the joint estimation of motion, moving object, and object intensity map. In these approaches the identification of one of the unknowns improves the capability to extract the other as well. Hence, combined methods turn out to be very powerful approaches. Indeed, fundamental tasks in image processing are highly interdependent: Registration of image morphology significantly benefits from previous denoising and structure segmentation. On the other hand, combined information of different image modalities makes shape segmentation significantly more robust. Furthermore, robustness in motion extraction of shapes can be significantly enhanced via a coupling with the detection of edge surfaces in space time and a corresponding feature sensitive space time smoothing. Furthermore, one of the key tools throughout most of the methods to be presented is nonlinear elasticity based on hyperelastic and polyconvex energy functionals. Based on first principles from continuum mechanics this allows a flexible description of shape correspondences and in many cases enables to establish existence results and one-to-one mapping properties. Numerical experiments underline the robustness of the presented methods and show applications on medical images and biological experimental data. This chapter is based on a couple of recent articles (Bar et al., A variational framework for simultaneous motion estimation and restoration of motion-blurred video, 2007; Litke et al., An image processing approach to surface matching, 2005; Droske et al., Comput. Vis. Sci. Online First, 2008; Droske and Rumpf, SIAM Appl Math 64(2):668–687, 2004; Droske and Rumpf, IEEE Trans Pattern Anal Mach Intell 29(12):2181–2194, 2007; Rumpf and Wirth, SIAM J Imag Sci, 2008) published by the author together with Leah Bar, Benjamin Berkels, Marc Droske, Nathan Litke, Wolfgang Ring, Guillermo Sapiro, Peter Schröder, and Benedikt Wirth.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 44.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 59.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. L. Alvarez, F. Guichard, P.-L. Lions, J.-M. Morel, Axioms and fundamental equations of image processing. Arch. Ration. Mech. Anal. 123(3), 199–257 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  2. L. Ambrosio, V.M. Tortorelli, Approximation of functionals depending on jumps by elliptic functionals via Γ-convergence. Comm. Pure Appl. Math. 43, 999–1036 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  3. L. Ambrosio, V.M. Tortorelli, On the approximation of free discontinuity problems. Boll. dell’Unione Matematica Ital. Sez. B 6(7), 105–123 (1992)

    MathSciNet  MATH  Google Scholar 

  4. L. Ambrosio, N. Fusco, D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems. Oxford Mathematical Monographs (Oxford University Press, New York, 2000)

    MATH  Google Scholar 

  5. T.J. Baker, Three dimensional mesh generation by triangulation of arbitrary point sets, in 8th Computational Fluid Dynamics Conference, vol. 1124-CP, Honolulu, 9–11 June 1987, pp. 255–271

    Google Scholar 

  6. J.M. Ball, Convexity conditions and existence theorems in nonlinear elasticity. Arch. Ration. Mech. Anal. 63, 337–403 (1977)

    Article  MATH  Google Scholar 

  7. J.M. Ball, Global invertibility of Sobolev functions and the interpenetration of matter. Proc. R. Soc. Edinb. 88A, 315–328 (1988)

    Google Scholar 

  8. L. Bar, B. Berkels, M. Rumpf, G. Sapiro, A variational framework for simultaneous motion estimation and restoration of motion-blurred video, in Eleventh IEEE International Conference on Computer Vision (ICCV’07) (2007)

    Google Scholar 

  9. K.K. Bhatia, J.V. Hajnal, B.K. Puri, A.D. Edwards, D. Rueckert, Consistent groupwise non–rigid registration for atlas construction, in IEEE International Symposium on Biomedical Imaging: Nano to Macro, vol. 1 (2004), pp. 908–911

    Google Scholar 

  10. B. Bourdin, Image segmentation with a finite element method. Technical Report Nr. 00-14, Institut Galilée Université Paris Nord, UCLA CAM, 2000; Article: Math. Model. Numer. Anal. 33(2), 229–244 (1999)

    Google Scholar 

  11. M. Bro-Nielsen, C. Gramkow, Fast Fluid Registration of Medical Images (Springer, Berlin, 1996), pp. 267–276

    Google Scholar 

  12. L.G. Brown, A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)

    Article  Google Scholar 

  13. T. Chan, L. Vese, An active contour model without edges, in Scale-Space Theories in Computer Vision. Second International Conference, Scale-Space ’99, Corfu, Greece, September 1999, ed. by M. Nielsen, P. Johansen, O.F. Olsen, J. Weickert. Lecture Notes in Computer Science, vol. 1682 (Springer, New York, 1999), pp. 141–151

    Google Scholar 

  14. G. Charpiat, O. Faugeras, R. Keriven, P. Maurel, Distance-based shape statistics, in Acoustics, Speech and Signal Processing, 2006 (ICASSP 2006), vol. 5 (2006). doi: 10.1109/ICASSP.2006.1661428

    Google Scholar 

  15. G. Charpiat, O. Faugeras, R. Keriven, Approximations of shape metrics and application to shape warping and empirical shape statistics. Found. Comut. Math. 5(1), 1–58 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  16. G.E. Christensen, S.C. Joshi, M.I. Miller, Volumetric transformations of brain anatomy. IEEE Trans. Med. Imag. 16(6), 864–877 (1997)

    Article  Google Scholar 

  17. G.E. Christensen, R.D. Rabbitt, M.I. Miller, A deformable neuroanatomy textbook based on viscous fluid mechanics, in Proceedings of the 27th Annual Conference Information Science and Systems, ed. by J. Prince, T. Runolfsson (1993), pp. 211–216. doi: 10.1109/ICASSP.2006.1661428

    Google Scholar 

  18. G.E. Christensen, M.I. Miller, R.D. Rabbitt, Deformable templates using large deformation kinematics. IEEE Trans. Image Process. 5(10), 1435–1447 (1996)

    Article  Google Scholar 

  19. P.G. Ciarlet, Three-Dimensional Elasticity (Elsevier, Amsterdam, 1988)

    MATH  Google Scholar 

  20. P.G. Ciarlet, J. Nečas, Injectivity and self-contact in nonlinear elasticity. Arch. Ration. Mech. Anal. 97(3), 171–188 (1987)

    Article  Google Scholar 

  21. U. Clarenz, M. Droske, M. Rumpf, Towards fast non–rigid registration, in Inverse Problems, Image Analysis and Medical Imaging, AMS Special Session Interaction of Inverse Problems and Image Analysis, vol. 313 (American Mathematical Society, Providence, 2002), pp. 67–84

    Google Scholar 

  22. U. Clarenz, N. Litke, M. Rumpf, Axioms and variational problems in surface parameterization. Comput. Aided Geom. Des. 21(8), 727–749 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  23. T.F. Cootes, C.J. Taylor, D.H. Cooper, J. Graham, Active shape models—their training and application. Comput. Vis. Image Underst. 61(1), 38–59 (1995)

    Article  Google Scholar 

  24. D. Cremers, S. Soatto, Motion competition: a variational approach to piecewise parametric motion segmentation. Int. J. Comput. Vis. 85(1), 115–132 (2004)

    Google Scholar 

  25. B. Dacorogna, Direct Methods in the Calculus of Variations (Springer, New York, 1989)

    Book  MATH  Google Scholar 

  26. C.A. Davatzikos, R.N. Bryan, J.L. Prince, Image registration based on boundary mapping. IEEE Trans. Med. Imag. 15(1), 112–115 (1996)

    Article  Google Scholar 

  27. M.C. Delfour, J.P. Zolésio, Geometries and Shapes: Analysis, Differential Calculus and Optimization. Advances in Design and Control, vol. 4 (SIAM, Philadelphia, 2001)

    Google Scholar 

  28. M. Droske, W. Ring, A Mumford-Shah level-set approach for geometric image registration. SIAM Appl. Math. (2008, to appear)

    Google Scholar 

  29. M. Droske, M. Rumpf, A variational approach to non-rigid morphological registration. SIAM J. Appl. Math. 66, 2127–2148 (electronic) (2006). doi:10.1137/050630209. http://dx.doi.org/10.1137/050630209

  30. M. Droske, M. Rumpf, Multi scale joint segmentation and registration of image morphology. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2181–2194 (2007)

    Article  Google Scholar 

  31. M. Droske, W. Ring, M. Rumpf, Mumford-Shah based registration: a comparison of a level set and a phase field approach. Comput. Vis. Sci. Online First (2008)

    Google Scholar 

  32. P. Favaro, S. Soatto, A variational approach to scene reconstruction and image segmentation from motion-blur cues, in 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04), vol. 1 (2004), pp. 631–637

    Google Scholar 

  33. O. Féron, A. Mohammad-Djafari, Image fusion and unsupervised joint segmentation using a HMM and MCMC algorithms. J. Electron. Imag. 023014-1, 023014-12 (2005)

    Article  Google Scholar 

  34. T. Fletcher, S. Venkatasubramanian, S. Joshi, Robust statistics on Riemannian manifolds via the geometric median, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York, 2008)

    Google Scholar 

  35. I. Fonseca, S. Müller, Quasi-convex integrands and lower semicontinuity in L 1. SIAM J. Math. Anal. 23(5), 1081–1098 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  36. M. Fuchs, B. Jüttler, O. Scherzer, H. Yang, Shape metrics based on elastic deformations. Forschungsschwerpunkt S92, Industrial Geometry 71, Universität Innsbruck, Innsbruck, 2008

    Google Scholar 

  37. E. Grinspun, A.H. Hirani, M. Desbrun, P. Schröder, Discrete shells, in Eurographics/SIGGRAPH Symposium on Computer Animation (Eurographics Association, 2003)

    Google Scholar 

  38. X. Gu, B.C. Vemuri, Matching 3D shapes using 2D conformal representations, in Medical Image Computing and Computer-Assisted Intervention 2004. Lecture Notes in Computer Science, vol. 3216 (2004), pp.771–780

    Article  Google Scholar 

  39. W. Hackbusch, S.A. Sauter, Composite finite elements for problems containing small geometric details. Part II: implementation and numerical results. Comput. Vis. Sci. 1(1), 15–25 (1997)

    MATH  Google Scholar 

  40. W. Hackbusch, S.A. Sauter, Composite finite elements for the approximation of PDEs on domains with complicated micro-structures. Numer. Math. 75, 447–472 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  41. W. Hackbusch, Iterative Solution of Large Sparse Systems of Equations. Applied Mathematical Sciences, vol. 95 (Springer, New York, 1994)

    Google Scholar 

  42. M. Hintermüller, W. Ring, An inexact Newton-cg-type active contour approach for the minimization of the Mumford-Shah functional. J. Math. Imag. Vis. 20(1–2), 19–42 (2004)

    Article  Google Scholar 

  43. S. Joshi, B. Davis, M. Jomier, G. Gerig, Unbiased diffeomorphic atlas construction for computational anatomy. NeuroImage 23(Suppl. 1), S151–S160 (2004)

    Article  Google Scholar 

  44. T. Kapur, L. Yezzi, L. Zöllei, A variational framework for joint segmentation and registration. IEEE CVPR - Mathematical Methods in Biomedical Image Analysis (IEEE, New York, 2001), pp. 44–51

    Google Scholar 

  45. S.L. Keeling, W. Ring, Medical image registration and interpolation by optical flow with maximal rigidity. J. Math. Imag. Vis. 23(1), 47–65 (2005)

    Article  MathSciNet  Google Scholar 

  46. D.G. Kendall, Shape manifolds, procrustean metrics, and complex projective spaces. Bull. Lond. Math. Soc. 16, 81–121 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  47. V. Kraevoy, A. Sheffer, Cross-parameterization and compatible remeshing of 3D models. ACM Trans. Graph. 23(3), 861–869 (2004)

    Article  Google Scholar 

  48. A. Lee, D. Dobkin, W. Sweldens, P. Schröder, Multiresolution mesh morphing, in Proceedings of SIGGRAPH 99. Computer Graphics Proceedings, Annual Conference Series, August 1999, pp. 343–350

    Google Scholar 

  49. N. Litke, M. Droske, M. Rumpf, P. Schröder, An image processing approach to surface matching, in Third Eurographics Symposium on Geometry Processing, ed. by M. Desbrun, H. Pottmann (2005), pp. 207–216

    Google Scholar 

  50. J.E. Marsden, T.J.R. Hughes, Mathematical Foundations of Elasticity (Prentice-Hall, Englewood Cliffs, 1983)

    MATH  Google Scholar 

  51. M.I. Miller, L. Younes, Group actions, homeomorphisms and matching: a general framework. Int. J. Comput. Vis. 41(1–2), 61–84 (2001)

    Article  MATH  Google Scholar 

  52. M.I. Miller, A. Trouvé, L. Younes, On the metrics and Euler-Lagrange equations of computational anatomy. Annu. Rev. Biomed. Eng. 4, 375–405 (2002)

    Article  Google Scholar 

  53. J.M. Morel, S. Solimini, Variational Models in Image Segmentation (Birkäuser, Basel, 1994)

    Google Scholar 

  54. C. Morrey, Quasi-convexity and lower semicontinuity of multiple integrals. Pac. J. Math. 2, 25–53 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  55. C. Morrey, Multiple Integrals in the Calculus of Variations. Grundlehren der mathematischen Wissenschaften, vol.130 (Springer, New-York, 1966)

    Google Scholar 

  56. D. Mumford, J. Shah, Optimal approximation by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42, 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  57. R.W. Ogden, Non-Linear Elastic Deformations (Wiley, London, 1984)

    Google Scholar 

  58. S.J. Osher, N. Paragios, Geometric Level Set Methods in Imaging, Vision and Graphics (Springer, New York, 2003)

    MATH  Google Scholar 

  59. S.J. Osher, J.A. Sethian, Fronts propagating with curvature dependent speed: algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79, 12–49 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  60. E. Praun, W. Sweldens, P. Schröder, Consistent mesh parameterizations. In Proceedings of ACM SIGGRAPH 2001. Computer Graphics Proceedings, Annual Conference Series, August 2001, pp. 179–184

    Google Scholar 

  61. L. Rudin, S. Osher, E. Fatemi, Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  62. D. Rueckert, A.F. Frangi, J.A. Schnabel, Automatic construction of 3D statistical deformation models using nonrigid registration, in Medical Image Computing and Computer-Assisted Intervention (MICCAI), ed. by W. Niessen, M. Viergever. Lecture Notes in Computer Science, vol. 2208 (2001), pp. 77–84

    Google Scholar 

  63. M. Rumpf, B. Wirth, A nonlinear elastic shape averaging approach. SIAM J. Imag. Sci. 2, 800–833. doi:10.1137/080738337

    Google Scholar 

  64. G. Sapiro, Geometric Partial Differential Equations and Image Analysis (Cambridge University Press, Cambridge, 2001)

    Book  MATH  Google Scholar 

  65. J. Schreiner, A. Asirvatham, E. Praun, H. Hoppe, Inter-surface mapping. ACM Trans. Graph. 23(3), 870–877 (2004)

    Article  Google Scholar 

  66. J. Sokołowski, J.-P. Zolésio, Introduction to Shape Optimization. Shape Sensitivity Analysis (Springer, Berlin, 1992)

    Book  MATH  Google Scholar 

  67. P.A. van den Elsen, E.-J.J. Pol, M.A. Viergever, Medical image matching: a review with classification. IEEE Eng. Med. Biol. 12, 26–39 (1993)

    Article  Google Scholar 

  68. V. Šverák, Regularity properties of deformations with finite energy. Arch. Ration. Mech. Anal. 100, 105–127 (1988)

    Article  MATH  Google Scholar 

  69. P.P. Wyatt, J.A. Noble, MAP MRF joint segmentation and registration, in Medical Image Computing and Computer–Assisted Intervention (MICCAI), ed. by T. Dohi, R. Kikinis. Lecture Notes in Computer Science, vol. 2488 (2002), pp. 580–587

    Google Scholar 

  70. A. Yezzi, L. Zöllei, T. Kapur, A variational framework for integrating segmentation and registration through active contours. Med. Image Anal. 7(2), 171–185 (2003)

    Article  Google Scholar 

  71. Y.-N. Young, D. Levy, Registration-based morphing of active contours for segmentation of ct scans. Math. Biosci. Eng. 2(1), 79–96 (2005)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The author is grateful to Werner Bautz, radiology department at the university hospital Erlangen, Germany, for providing CT data of kidneys, as well as to Heiko Schlarb from Adidas, Herzogenaurach, Germany, for providing 3D scans of feet, and to Bruno Wirth, urology department at the Hospital zum Hl. Geist, Kempen, Germany, for providing thorax CT scans. Furthermore, the author thanks Stan Osher for pointing to the issue of elastic shape averaging and Marc Droske for discussion about the phase field approach. Finally, he acknowledges Helene Horn for her help in the careful preparation of the manuscript.

Author information

Authors and Affiliations

  1. University of Bonn, Bonn, Germany

    Martin Rumpf

Authors
  1. Martin Rumpf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Rumpf .

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Rumpf, M. (2013). Variational Methods in Image Matching and Motion Extraction. In: Level Set and PDE Based Reconstruction Methods in Imaging. Lecture Notes in Mathematics(), vol 2090. Springer, Cham. https://doi.org/10.1007/978-3-319-01712-9_3

Download citation

Publish with us

Policies and ethics

Search

Navigation