International Journal of Computer Vision

, Volume 61, Issue 1, pp 31–54

Adaptive Grid Refinement Procedures for Efficient Optical Flow Computation

  • Joan Condell
  • Bryan Scotney
  • Philip Morrow
Article

Abstract

Two approaches are described that improve the efficiency of optical flow computation without incurring loss of accuracy. The first approach segments images into regions of moving objects. The method is based on a previously defined Galerkin finite element method on a triangular mesh combined with a multiresolution segmentation approach for object flow computation. Images are automatically segmented into subdomains of moving objects by an algorithm that employs a hierarchy of mesh coarseness for the flow computation, and these subdomains are reconstructed over a finer mesh on which to recompute flow more accurately. The second approach uses an adaptive mesh in which the resolution increases where motion is found to occur. Optical flow is computed over a reasonably coarse mesh, and this is used to construct an optimal adaptive mesh in a way that is different from the gradient methods reported in the literature. The finite element mesh facilitates a reduction in computational effort by enabling processing to focus on particular objects of interest in a scene (i.e. those areas where motion is detected). The proposed methods were tested on real and synthetic image sequences, and promising results are reported.

adaptive grids Delaunay algorithm inverse finite elements motion estimation optical flow triangular meshes 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aisbett, J. 1989.Optical-flow with an intensity-weighted smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Computer Society Press,11(5):512-522.Google Scholar
  2. Anandan, P. 1989.A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision,2:283-310.Google Scholar
  3. Axelsson, O. and Barker, V.A. 2001.Finite element solution of boundary value problems: Theory and computation. SIAM Classics in Applied Mathematics, 35.Google Scholar
  4. Barron, J.L., Fleet, D.J., and Beauchemin, S.S. 1994.Performance of optical flowtechniques. International Journal of Computer Vision,12(l):43-77.Google Scholar
  5. Bartolini, F. and Piva, A. 1997.Enhancement of the Horn and Schunck optic flow algorithm by means of median filters. In IEEE Proceedings Thirteenth International Conference on Digital Signal Processing (DSP 1997) IEEE Computer Society Press, 1,2:503-506.Google Scholar
  6. Battiti, R., Amaldi, E., and Koch, C. 1991.Computing optical flow across multiple scales: An adaptive coarse-to-fine strategy. International Journal of Computer Vision,6(2):133-145.Google Scholar
  7. Beauchemin, S.S. and Barron, J.L. 1995.The computation of optical flow. ACM Computing Surveys, 27(3):433-467Google Scholar
  8. Benayoun, S. and Ayache, N. 1998.Dense non-rigid motion estimation in sequences of medical images using differential constraints. International Journal of Computer Vision,26(1):25-40.Google Scholar
  9. Bergen, J.R., Burt, P.J., Hingorani, R., and Peleg, S. 1990.Computing two motions from three frames. In Proceedings of the Third International Conference on Computer Vision (ICCV 1990)Osaka, Japan, 27-32.Google Scholar
  10. Black, M.J. and Anandan, P. 1996.The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding,63(1):75-104.Google Scholar
  11. Celasun, I., Tekalp, A.M., Gokcetekin, M.H., and Harmanci, D.M. 2001.2-D mesh-based video object segmentation and tracking with occlusion resolution. Signal Processing: Image Communication,16:949-962.Google Scholar
  12. Choi, J.G. and Kim, S.D. 1996.Multi-stage segmentation of optical flow field. Signal In Processing,54:109-118.Google Scholar
  13. Cohen, I. and Herlin, I. 1999.Non uniform multiresolution method for optical flow and phase portrait models: Environmental applications. International Journal of Computer Vision,33(l):1-22Google Scholar
  14. Condell, J.V., Scotney, B.W., and Morrow, P.J. 2001a. Estimation of motion through inverse finite element methods with triangular meshes. In Proceedings of the Ninth International Conference on Computer Analysis of Images and Patterns (CAIP 2001),Warsaw, Poland, Lecture Notes on Computer Science, vol. 2124, pp. 333-340.Google Scholar
  15. Condell, J.V., Scotney, B.W., and Morrow, P.J. 2001b. Motion estimation through inverse finite element methods based on triangular meshes: Towards automatic region detection. In Proceedings of the Irish Machine Vision and Image Processing Conference (IMVIP 2001)Maynooth, Ireland, pp. 53-60.Google Scholar
  16. Devlaminck, V. 1995.Finite element method for non-rigid motion estimation. In Proceedings of the SPIE International Society of Optical Engineering,2588:312-319.Google Scholar
  17. Enkelmann, W. 1988.Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences. Computer Vision, Graphics and Image Processing,43(2):150-177.Google Scholar
  18. Fennema, C.L. and Thompson, W.B. 1979.Velocity determination in scenes containing several moving objects. Computer Graphics and Image Processing,9(4):301-315.Google Scholar
  19. Fleet, D.J. and Jepson, A.D. 1989.Computation of normal velocity from local phase information. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (GVPR 1989), San Diego, California, USA, pp. 379-386.Google Scholar
  20. Garcia, M.A., Vintimilla, B.X., and Sappa, A.D. 1999.Approximation of intensity images with adaptive triangular meshes: Towards a processable compressed representation. In Proceedings of the Third Irish Machine Vision and Image Processing Conference (IMVIP 1999), Dublin, Ireland, pp. 241-249.Google Scholar
  21. Giachetti, A. and Torre, V. 1996.Refinement of optical flow estimation and detection of motion edges. In Proceedings of the Fourth European Conference on Computer Vision (ECCV 1996), Cambridge, UK, pp. 151-160.Google Scholar
  22. Goh, W.B. and Martin, G.R. 1994.Model-based multiresolution motion estimation in noisy images. CVGIP Image Understanding,59(3):307-319.Google Scholar
  23. Graham, J.V., Scotney, B.W., and Morrow, P.J. 2000a. Inverse finite element formulations for the computation of optical flow. In Proceedings of the Fourth Irish Machine Vision and Image Processing Conference (IMVIP 2000)Belfast, Northern Ireland, pp. 59-66.Google Scholar
  24. Graham, J.V., Scotney, B.W., and Morrow, P.J. 2000b. Evaluation of inverse finite element techniques for gradient based motion estimation. In Proceedings of the Third IMA Conference on Imaging and Digital Image Processing, Leicester, UK.Google Scholar
  25. Hata, N., Nabavi, A. Warfield, S., Wells, W., Kikinis, R., and Jolesz, F.A. 1999.A volumetric optical flow method for measurement of brain deformation from intraoperative magnetic resonance images. In Second International Conference on Medical Image Computing and Computer-Assisted Intervention, (MICCAI 1999), Cambridge, UK, Lecture Notes in Computer Science, vol. 1679, pp. 928-933.Google Scholar
  26. Heeger, D.J. 1987.Model for the extraction of image flow. Journal of the Optical Society of America A-Optics Image Science and Vision,4(8):1455-1471.Google Scholar
  27. Horn, B.K.P, and Schunck, B.G. Determining optical flow. Artificial Intelligence,17:185-203.Google Scholar
  28. Hwang, S.H. and Lee, S.U. 1993.A hierarchical optical flow estimation algorithm based on the interlevel motion smoothness constraint. Pattern Recognition,26(6):939-952.Google Scholar
  29. Illgner, K. and Müller, F. 1997.Image segmentation using motion estimation. Time-Varying Image Processing and Moving Object Recognition,4:238-243.Google Scholar
  30. Kirchner, H. and Niemann, H. 1992.Finite element method for determination of optical flow. Pattern Recognition Letters,13:131-141.Google Scholar
  31. Lucas, B. and Kanade, T. 1981.An iterative image registration technique with an application to stereo vision. In Proceedings of the Seventh International Joint Conference on Artificial Intelligence (IJCAI 1981),Vancouver, British Columbia, Canada, pp. 674-679.Google Scholar
  32. Memin, E. and Perez, P. 1998.A multigrid approach for hierarchical motion estimation. In Sixth International Conference on Computer Vision (ICCV1998), Bombay, India, pp. 933-938.Google Scholar
  33. Moulin, P., Krishnamurthy, R., and Woods, J.W. 1997.Multiscale modeling and estimation of motion fields for video coding. In IEEE Transactions on Image Processing,6(12):1606-1620.Google Scholar
  34. Moulin, P. and Loui, A. 1993.Application of a multiresolution optical-flow-based method for motion estimation to video coding. In Proceedings of the 1993 IEEE International Symposium on Circuits and Systems,1:1-4.Google Scholar
  35. Nagel, H.H. 1983.Displacement vectors derived from second order intensity variations in image sequences. Computer Vision, Graphics and Image Processing,21(1):85-117.Google Scholar
  36. Odobez, J.M. and Bouthemy, P. 1994.Robust multiresolution estima-tion of parametric motion models in complex image sequences. In Proceedings of the Seventh European Conference on Signal Processing, (EUSIPCO 1994), Edinburgh, Scotland, pp. 411-415.Google Scholar
  37. Ong, E.P. and Spann, M. 1999.Robust optical flow computation based on least-median-of-squares regression. International Journal of Computer Vision,31(1):51-82.Google Scholar
  38. Schnörr, C. 1991.Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class. International Journal of Computer Vision,6(1):25-38.Google Scholar
  39. Scotney, B.W. 1991.Construction and analysis of petrov-galerkin approximations for convection-dominated flows. In Proceedings Thirteenth World Congress on Computation and Applied Mathematics, Dublin, IMACS, R.Vichnevetsky and J.J.H. Miller, (Eds.), pp. 505-507.Google Scholar
  40. Szeliski, R. and Shum, H.Y. 1996.Motion estimation with quadtree splines. IEEE Transactions on Pattern Analysis and Machine In-telligence,18(12):1199-1210.Google Scholar
  41. Terzopoulos, D. 1986.Image analysis using multigrid relaxation methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8(2):129-139.Google Scholar
  42. Wang, Y. and Lee, O. 1996.Use of two-dimensional deformable mesh structures for video coding, Part I-The synthesis problem: Mesh-based function approximation and mapping. IEEE Transactions on Circuits and Systems for Video Technology.6(6):636-646.Google Scholar
  43. Weber, J. and Malik, J. 1995.Robust computation of optical flow in a multiscale differential framework. International Journal of Computer Vision,14(1):67-81.Google Scholar
  44. Weickert, J. and Schnörr, C. 2001.A theoretical framework for convex regularizers in pde-based computation of image motion. International Journal of Computer Vision, 45(3):245-264.Google Scholar
  45. Wu, S.F and Kittler, J. 1993.A gradient-based method for general motion estimation and segmentation. Journal of Visual Communication and Image Representation,4(1):25-38.Google Scholar
  46. Xie, K., Eycken, L.V., and Oosterlinck, A. 1996.Hierarchical motion estimation with smoothness constraint and postprocessing. Optical Engineering,35(1):145-155.Google Scholar
  47. Yaacobson, F.S. and Givoli, D. 1998.An adaptive finite element procedure for the image segmentation problem. Communications of Numerical Methods in Engineering,14:621-632.Google Scholar
  48. Yang, Y., Wernick, M.N., and Brankov, J.G. 2001.A fast algorithm for accurate content-adaptive mesh generation. IEEE International Conference on Image Processing (ICIP 2001), Thessaloniki, Greece, pp. 868-871.Google Scholar

Copyright information

© Kluwer Academic Publishers 2005

Authors and Affiliations

  • Joan Condell
    • 1
  • Bryan Scotney
    • 1
  • Philip Morrow
    • 1
  1. 1.School of Computing and Intelligent SystemsFaculty of Engineering, University of Ulster, Magee CollegeLondonderryNorthern Ireland

Personalised recommendations