Skip to main content
SpringerLink
Log in

Particle Video: Long-Range Motion Estimation Using Point Trajectories

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper describes a new approach to motion estimation in video. We represent video motion using a set of particles. Each particle is an image point sample with a long-duration trajectory and other properties. To optimize particle trajectories we measure appearance consistency along the particle trajectories and distortion between the particles. The resulting motion representation is useful for a variety of applications and cannot be directly obtained using existing methods such as optical flow or feature tracking. We demonstrate the algorithm on challenging real-world videos that include complex scene geometry, multiple types of occlusion, regions with low texture, and non-rigid deformations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alvarez, L., Deriche, R., Papadopoulo, T., & Sanchez, J. (2002). Symmetrical dense optical flow estimation with occlusion detection. In ECCV (pp. 721–735).

  • Amiaz, T., & Kiryati, N. (2005). Dense discontinuous optical flow via contour-based segmentation. In ICIP (pp. 1264–1267).

  • Baker, S., & Matthews, I. (2004). Lucas-Kanade 20 years on: A unifying framework. International Journal of Computer Vision, 56(3), 221–255.

    Article  Google Scholar 

  • Baker, S., Scharstein, D., Lewis, J. P., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. In ICCV.

  • Barrett, R., Berry, M., Chan, T. F., Demmel, J., Donato, J. M., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., & der Vorst, H. V. (1994). Templates for the solution of linear systems: building blocks for iterative methods. Philadelphia: SIAM.

    Google Scholar 

  • Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994). Performance of optical flow techniques. International Journal of Computer Vision, 12(1), 43–77.

    Article  Google Scholar 

  • Beauchemin, S. S., & Barron, J. L. (1995). The computation of optical flow. ACM Computing Surveys, 27(3), 433–467.

    Article  Google Scholar 

  • Bergen, J. R., Anandan, P., Hanna, K. J., & Hingorani, R. (1992). Hierarchical model-based motion estimation. In ECCV (pp. 237–252).

  • Black, M., & Anandan, P. (1991). Robust dynamic motion estimation over time. In CVPR (pp. 296–302).

  • Black, M. J. (1994). Recursive non-linear estimation of discontinuous flow fields. In ECCV (pp. 138–144).

  • Black, M. J., & Anandan, P. (1996). The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1), 75–104.

    Article  Google Scholar 

  • Brand, M. (2001). Morphable 3D models from video. In CVPR (pp. 456–463).

  • Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In ECCV (pp. 25–36).

  • Chin, T. M., Karl, W. C., & Willsky, A. S. (1994). Probabilistic and sequential computation of optical flow using temporal coherence. IEEE Transactions on Image Processing, 3(6), 773–788.

    Article  Google Scholar 

  • Elad, M., & Feuer, A. (1998). Recursive optical flow estimation–adaptive filtering approach. Visual Communication and Image Representation, 9(2), 119–138.

    Article  Google Scholar 

  • Fusiello, A., Trucco, E., Tommasini, T., & Roberto, V. (1999). Improving feature tracking with robust statistics. Pattern Analysis and Applications, 2(4), 312–320.

    Article  Google Scholar 

  • Goldman, D. B., Curless, B., Salesin, D., & Seitz, S. M. (2007). Interactive video object annotation (Technical Report UW-CSE-2007-04-01). University of Washington.

  • Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Haussecker, H. W., & Fleet, D. J. (2001). Computing optical flow with physical models of brightness variation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 661–673.

    Article  Google Scholar 

  • Irani, M. (1999). Multi-frame optical flow estimation using subspace constraints. In ICCV (pp. 626–633).

  • Lischinski, D. (1994). Incremental Delaunay triangulation. In Graphics gems IV (pp. 47–59). San Diego: Academic Press.

    Google Scholar 

  • Liu, C., Torralba, A., Freeman, W. T., Durand, F., & Adelson, E. H. (2005). Motion magnification. ACM Transactions on Graphics, 24(3), 519–526.

    Article  Google Scholar 

  • Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In International joint conference on artificial intelligence (pp. 674–679).

  • Murray, D. W., & Buxton, B. F. (1987). Scene segmentation from visual motion using global optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(2), 220–228.

    Article  Google Scholar 

  • Papenberg, N., Bruhn, A., Brox, T., Didas, S., & Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2), 141–158.

    Article  Google Scholar 

  • Sand, P. (2006). Long-range video motion estimation using point trajectories. PhD thesis, MIT.

  • Sand, P., & Teller, S. (2006). Particle video: Long-range motion estimation using point trajectories. In CVPR (pp. 2195–2202).

  • Sawhney, H. S., Guo, Y., Hanna, K., Kumar, R., Adkins, S., & Zhou, S. (2001). Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences. In SIGGRAPH (pp. 451–460).

  • Shi, J., & Malik, J. (1998). Motion segmentation and tracking using normalized cuts. In ICCV (pp. 1154–1160).

  • Shi, J., & Tomasi, C. (1994). Good features to track. In CVPR (pp. 593–600).

  • Silva, C., & Santos-Victor, J. (2001). Motion from occlusions. Robotics and Autonomous Systems, 35(3–4), 153–162.

    Article  MATH  Google Scholar 

  • Strecha, C., Fransens, R., & Gool, L. V. (2004). A probabilistic approach to large displacement optical flow and occlusion detection. In Statistical methods in video processing (pp. 71–82).

  • Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319–2322.

    Article  Google Scholar 

  • Thompson, W. (1998). Exploiting discontinuities in optical flow. International Journal of Computer Vision, 30(3), 163–174.

    Article  Google Scholar 

  • Weickert, J., Bruhn, A., Papenberg, N., & Brox, T. (2004). Variational optic flow computation: From continuous models to algorithms. In International workshop on computer vision and image analysis (pp. 1–6).

  • Xiao, J., Cheng, H., Sawhney, H., Rao, C., & Isnardi, M. (2006). Bilateral filtering-based optical flow estimation with occlusion detection. In ECCV (pp. 211–224).

  • Zitnick, C. L., Jojic, N., & Kang, S. B. (2005). Consistent segmentation for optical flow estimation. In ICCV (pp. 1308–1315).

Download references

Author information

Authors and Affiliations

  1. MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA, 02139, USA

    Peter Sand & Seth Teller

Authors
  1. Peter Sand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seth Teller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Sand.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sand, P., Teller, S. Particle Video: Long-Range Motion Estimation Using Point Trajectories. Int J Comput Vis 80, 72–91 (2008). https://doi.org/10.1007/s11263-008-0136-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-008-0136-6

Keywords

Search

Navigation