Optical Flow on Evolving Surfaces with Space and Time Regularisation

  • Clemens Kirisits
  • Lukas F. Lang
  • Otmar Scherzer


We extend the concept of optical flow with spatiotemporal regularisation to a dynamic non-Euclidean setting. Optical flow is traditionally computed from a sequence of flat images. The purpose of this paper is to introduce variational motion estimation for images that are defined on an evolving surface. Volumetric microscopy images depicting a live zebrafish embryo serve as both biological motivation and test data.


Biomedical imaging Computer vision Evolving surfaces Optical flow Spatiotemporal regularisation Variational methods 



We thank Pia Aanstad from the University of Innsbruck for sharing her biological insight and for kindly providing the microscopy data. This work has been supported by the Vienna Graduate School in Computational Science (IK I059-N) funded by the University of Vienna. In addition, we acknowledge the support by the Austrian Science Fund (FWF) within the national research networks “Photoacoustic Imaging in Biology and Medicine” (project S10505-N20, Reconstruction Algorithms for PAI) and “Geometry + Simulation” (project S11704, Variational Methods for Imaging on Manifolds).


  1. 1.
    Abràmoff, M.D., Viergever, M.A.: Computation and visualization of three-dimensional soft tissue motion in the orbit. IEEE Trans. Med. Imag. 21(4), 296–304 (2002)CrossRefGoogle Scholar
  2. 2.
    Amat, F., Myers, E.W., Keller, P.J.: Fast and robust optical flow for time-lapse microscopy using super-voxels. Bioinformatics 29(3), 373–380 (2013)CrossRefGoogle Scholar
  3. 3.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vision 92(1), 1–31 (November 2011)Google Scholar
  4. 4.
    Batchelor, G.K.: An Introduction to Fluid Dynamics. Cambridge Mathematical Library. Cambridge University Press, Cambridge (1999)zbMATHGoogle Scholar
  5. 5.
    Buibas, M., Yu, D., Nizar, K., Silva, G.A.: Mapping the spatiotemporal dynamics of calcium signaling in cellular neural networks using optical flow. Ann. Biomed. Eng. 38(8), 2520–2531 (2010)CrossRefGoogle Scholar
  6. 6.
    Cermelli, P., Fried, E., Gurtin, M.E.: Transport relations for surface integrals arising in the formulation of balance laws for evolving fluid interfaces. J. Fluid Mech. 544, 339–351 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  7. 7.
    Courant, R., Hilbert, D.: Methods of Mathematical Physics, vol. 1. Interscience Publishers Inc., New York, NY (1953)Google Scholar
  8. 8.
    Delpiano, J., Jara, J., Scheer, J., Ramírez, O.A., Ruiz-del Solar, J.: Performance of optical flow techniques for motion analysis of fluorescent point signals in confocal microscopy. Mach. Vis. Appl. 23(4), 675–689 (2012)CrossRefGoogle Scholar
  9. 9.
    do Carmo, M.P.: Differential Geometry of Curves and Surfaces. Prentice-Hall, London (1976)zbMATHGoogle Scholar
  10. 10.
    do Carmo, M.P.: Riemannian Geometry. Birkhäuser, Boston (1992)CrossRefzbMATHGoogle Scholar
  11. 11.
    Dziuk, G., Elliott, C.M.: Finite element methods for surface PDEs. Acta Numer. 22, 289–396 (2013)CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Revised English Edition Translated and Edited by Richard A. Silverman. Prentice-Hall Inc., Englewood Cliffs, NJ (1963)Google Scholar
  13. 13.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)CrossRefGoogle Scholar
  14. 14.
    Hubený J., Ulman V., Matula P.: Estimating large local motion in live-cell imaging using variational optical flow. In: VISAPP: Proceedigs of the Second International Conference on Computer Vision Theory and Applications, pp. 542–548. INSTICC, Lisbon (2007)Google Scholar
  15. 15.
    Imiya, A., Sugaya, H., Torii, A., Mochizuki, Y.: Variational analysis of spherical images. In: Gagalowicz, A., Philips, W. (eds.) Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, vol. 3681, pp. 104–111. Springer, Berlin (2005)CrossRefGoogle Scholar
  16. 16.
    Kimmel, C.B., Ballard, W.W., Kimmel, S.R., Ullmann, B., Schilling, T.F.: Stages of embryonic development of the zebrafish. Dev. Dyn. 203(3), 253–310 (1995)CrossRefGoogle Scholar
  17. 17.
    Kirisits, C., Lang, L.F., Scherzer, O.: Optical flow on evolving surfaces with an application to the analysis of 4D microscopy data. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM’13: Proceedings of the fourth International Conference on Scale Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, vol. 7893, pp. 246–257. Springer, Berlin (2013)Google Scholar
  18. 18.
    Kühnel, W.: Differential Geometry: Curves–Surfaces–Manifolds. Student Mathematical Library, vol. 16. American Mathematical Society, Providence, RI (2006). Translated from the 2003 German original by Bruce HuntGoogle Scholar
  19. 19.
    Lee, J.M.: Riemannian Manifolds: An Introduction to Curvature. Graduate Texts in Mathematics, vol. 176. Springer, New York (1997)zbMATHGoogle Scholar
  20. 20.
    Lee, J.M.: Introduction to Smooth Manifolds. Graduate Texts in Mathematics, vol. 218, 2nd edn. Springer, New York (2013)zbMATHGoogle Scholar
  21. 21.
    Lefèvre, J., Baillet, S.: Optical flow and advection on 2-Riemannian manifolds: a common framework. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1081–1092 (2008)CrossRefGoogle Scholar
  22. 22.
    Megason, S.G., Fraser, S.E.: Digitizing life at the level of the cell: high-performance laser-scanning microscopy and image analysis for in toto imaging of development. Mech. Dev. 120(11), 1407–1420 (2003)CrossRefGoogle Scholar
  23. 23.
    Melani, C., Campana, M., Lombardot, B., Rizzi, B., Veronesi, F., Zanella, C., Bourgine, P., Mikula, K., Peyriéras, N., Sarti, A.: Cells tracking in a live zebrafish embryo. In: Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2007), pp. 1631–1634 (2007)Google Scholar
  24. 24.
    Miura, K.: Tracking movement in cell biology. In: Rietdorf, J. (ed.) Microscopy Techniques. Advances in Biochemical Engineering/Biotechnology, vol. 95, pp. 267–295. Springer, Belin (2005)CrossRefGoogle Scholar
  25. 25.
    Mizoguchi, T., Verkade, H., Heath, J.K., Kuroiwa, A., Kikuchi, Y.: Sdf1/Cxcr4 signaling controls the dorsal migration of endodermal cells during zebrafish gastrulation. Development 135(15), 2521–2529 (2008)CrossRefGoogle Scholar
  26. 26.
    Quelhas, P., Mendonça, A.M., Campilho, A.: Optical flow based Arabidopsis Thaliana root meristem cell division detection. In: Campilho, A., Kamel, M. (eds.) Image Analysis and Recognition. Lecture Notes in Computer Science, vol. 6112, pp. 217–226. Springer, Berlin (2010)CrossRefGoogle Scholar
  27. 27.
    Schmid, B., Shah, G., Scherf, N., Weber, M., Thierbach, K., Campos Pérez, C., Roeder, I., Aanstad, P., Huisken, J.: High-speed panoramic light-sheet microscopy reveals global endodermal cell dynamics. Nat. Commun. 4, 2207 (2013)CrossRefGoogle Scholar
  28. 28.
    Torii, A., Imiya, A., Sugaya, H., Mochizuki, Y.: Optical flow computation for compound eyes: variational analysis of omni-directional views. In: De Gregorio, M., Di Maio, V., Frucci, M., Musio, C. (eds.) Brain, Vision, and Artificial Intelligence. Lecture Notes in Computer Science, vol. 3704, pp. 527–536. Springer, Berlin (2005)CrossRefGoogle Scholar
  29. 29.
    Warga, R.M., Nüsslein-Volhard, C.: Origin and development of the zebrafish endoderm. Development 126(4), 827–838 (1999)Google Scholar
  30. 30.
    Weickert, J., Bruhn, A., Brox, T., Papenberg, N.: A survey on variational optic flow methods for small displacements. In: Scherzer, O. (ed.) Mathematical Models for Registration and Applications to Medical Imaging. Mathematics in Industry, vol. 10, pp. 103–136. Springer, Berlin (2006)CrossRefGoogle Scholar
  31. 31.
    Weickert, J., Schnörr, C.: Variational optic flow computation with a spatio-temporal smoothness constraint. J. Math. Imaging Vis. 14, 245–255 (2001)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Clemens Kirisits
    • 1
  • Lukas F. Lang
    • 1
  • Otmar Scherzer
    • 1
    • 2
  1. 1.Computational Science CenterUniversity of ViennaViennaAustria
  2. 2.Radon Institute of Computational and Applied MathematicsAustrian Academy of SciencesLinzAustria

Personalised recommendations