Abstract
Most algorithms for dense 2D motion estimation assume pairs of images that are acquired with an idealized, infinitively short exposure time. In this work we compare two approaches that use an additional, motion-blurred image of a scene to estimate highly accurate, dense correspondence fields.
We consider video sequences that are acquired with alternating exposure times so that a short-exposure image is followed by a long-exposure image that exhibits motion-blur. For both motion estimation algorithms we employ an image formation model that relates the motion blurred image to two enframing short-exposure images. With this model we can decipher the motion information encoded in the long-exposure image, but also estimate occlusion timings which are a prerequisite for artifact-free frame interpolation. The first approach solves for the motion in a pointwise least squares formulation while the second formulates a global, total variation regularized problem. Both approaches are evaluated in detail and compared to each other and state-of-the-art motion estimation algorithms.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. IJCV 12(1), 43–77 (1994)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proc. ICCV, pp. 1–8. IEEE, Los Alamitos (2007)
Christmas, W.: Filtering requirements for gradient-based optical flow measurement. T-IP 9, 1817–1820 (2000)
Sellent, A., Eisemann, M., Goldlücke, B., Cremers, D., Magnor, M.: Motion field estimation from alternate exposure images. T-PAMI (to appear)
Kundur, D., Hatzinakos, D.: Blind image deconvolution. IEEE Signal Process Magazine 13, 43–64 (1996)
Xiao, J., Cheng, H., Sawhney, H., Rao, C., Isnardi, M.: Bilateral filtering-based optical flow estimation with occlusion detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 211–224. Springer, Heidelberg (2006)
Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. IJCV 80, 72–91 (2008)
Alvarez, L., Deriche, R., Papadopoulo, T., Sanchez, J.: Symmetrical dense optical flow estimation with occlusions detection. IJCV 75, 371–385 (2007)
Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. In: Proc. CVPR, pp. 1293–1300. IEEE, San Francisco (2010)
Sellent, A., Eisemann, M., Magnor, M.: Motion Field and Occlusion Time Estimation via Alternate Exposure Flow. In: Proc. ICCP. IEEE, Los Alamitos (2009)
Sellent, A., Eisemann, M., Goldlücke, B., Pock, T., Cremers, D., Magnor, M.: Variational optical flow from alternate exposure images. In: Proc. VMV, pp. 135–143 (2009)
Aggarwal, J., Nandhakumar, N.: On the computation of motion from sequences of images-a review. Proc. of the IEEE 76, 917–935 (1988)
Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. IJCV 2, 283–310 (1989)
Alvarez, L., Weickert, J., Sánchez, J.: Reliable estimation of dense optical flow fields with large displacements. IJCV 39, 41–56 (2000)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)
Lim, S., Apostolopoulos, J., Gamal, A.: Optical flow estimation using temporally oversampled video. T-IP 14, 1074–1087 (2005)
Black, M.J., Anandan, P.: Robust dynamic motion estimation over time. In: Proc. CVPR, pp. 296–302 (1991)
Mahajan, D., Huang, F., Matusik, W., Ramamoorthi, R., Belhumeur, P.: Moving gradients. In: Proc. SIGGRAPH. ToG, vol. 28, pp. 1–11. ACM, New York (2009)
Yitzhaky, Y., Kopeika, N.: Identification of blur parameters from motion blurred images. Graphical Models and Image Processing 59, 310–320 (1997)
Pao, T., Kuo, M.: Estimation of the point spread function of a motion-blurred object from autocorrelation. In: Proc. of SPIE, vol. 2501 (2003)
Rekleitis, I.M.: Optical flow recognition from the power spectrum of a single blurred image. In: Proc. ICIP, pp. 791–794. IEEE, Los Alamitos (1996)
Jia, J.: Single image motion deblurring using transparency. In: Proc. CVPR, pp. 1–8. IEEE Computer Society, Los Alamitos (2007)
Dai, S., Wu, Y.: Motion from blur. In: Proc.CVPR, pp. 1–8. IEEE, Los Alamitos (2008)
Wang, J., Cohen, M.F.: Image and video matting: A survey. Foundations and Trends in Computer Graphics and Vision 3, 97–175 (2007)
Fergus, R., Singh, B., Hertzmann, A., Roweis, S., Freeman, W.: Removing camera shake from a single photograph. ToG (2006)
Xu, L., Jia, J.: Two-phase kernel estimation for robust motion deblurring. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 157–170. Springer, Heidelberg (2010)
Bardsley, J., Jefferies, S., Nagy, J., Plemmons, R.: Blind iterative restoration of images with spatially-varying blur. Optics Express 14, 1767–1782 (2006)
Levin, A.: Blind motion deblurring using image statistics. Advances in Neural Information Processing Systems 19, 841–848 (2007)
Tico, M., Vehvilainen, M.: Estimation of motion blur point spread function from differently exposed image frames. In: Proc. of Eusipco, Florence, Italy (2006)
Yuan, L., Sun, J., Quan, L., Shum, H.Y.: Image deblurring with blurred/noisy image pairs. In: Proc. SIGGRAPH. ToG, vol. 26, pp. 1–8. ACM, New York (2007)
Lim, S., Silverstein, A.: Estimation and removal of motion blur by capturing two images with different exposures (2008)
Ben-Ezra, M., Nayar, S.: Motion-based motion deblurring. T-PAMI 26, 689 (2004)
Tai, Y., Du, H., Brown, M., Lin, S.: Image/video deblurring using a hybrid camera. In: Proc. CVPR, pp. 1–8. IEEE Computer Society, Los Alamitos (2008)
Rav-Acha, A., Peleg, S.: Restoration of multiple images with motion blur in differentdirections. In: Workshop on Appl. of Comp. V., pp. 22–28. IEEE, Los Alamitos (2000)
Rav-Acha, A., Peleg, S.: Two motion-blurred images are better than one. Pattern Recognition Letters 26, 311–317 (2005)
Cho, T., Levin, A., Durand, F., Freeman, W.: Motion blur removal with orthogonal parabolic exposures. In: Proc. ICCP, pp. 1–8 (2010)
Chen, W.G., Nandhakumar, N., Martin, W.N.: Image motion estimation from motion smear-a new computational model. T-PAMI 18 (1996)
Chen, W.G., Nandhakumar, N., Martin, W.N.: Estimating image motion from smear: a sensor system and extensions. In: Proc. ICIP, pp. 199–202. IEEE, Los Alamitos (1995)
Favaro, P., Soatto, S.: A variational approach to scene reconstruction and image segmentation from motion-blur cues. In: Proc. CVPR. IEEE, Los Alamitos (2004)
Agrawal, A., Xu, Y., Raskar, R.: Invertible motion blur in video. In: Proc. SIGGRAPH. ToG, vol. 28, pp. 1–8. ACM, New York (2009)
Dennis, J., Schnabel, R.: Numerical methods for unconstrained optimization and nonlinear equations. Prentice-Hall, Englewood Cliffs (1983)
Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. IJCV 59 (2004)
Forsythe, G.E., Malcolm, M.A., Moler, C.B.: Computer Methods for Mathematical Computations. Prentice-Hall, Englewood Cliffs (1976)
Tikhonov, A., Arsenin, V.: Solutions of Ill-Posed Problems. Winston, NY (1977)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)
Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Comp. V. and Img. Underst. 63, 75–104 (1996)
Chambolle, A.: An algorithm for total variation minimization and applications. Journal of Mathematical Image Visualization 20, 89–97 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Sellent, A., Eisemann, M., Magnor, M. (2011). Two Algorithms for Motion Estimation from Alternate Exposure Images. In: Cremers, D., Magnor, M., Oswald, M.R., Zelnik-Manor, L. (eds) Video Processing and Computational Video. Lecture Notes in Computer Science, vol 7082. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24870-2_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-24870-2_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24869-6
Online ISBN: 978-3-642-24870-2
eBook Packages: Computer ScienceComputer Science (R0)