Temporal Prediction and Spatial Regularization in Differential Optical Flow
In this paper we present an extension to the Bayesian formulation of multi-scale differential optical flow estimation by Simoncelli et. al.. We exploit the observation that optical flow is consistent in consecutive time frames and thus propagating information over time should improve the quality of the flow estimation. This propagation is formulated via insertion of additional Kalman filters that filter the flow over time by tracking the movement of each pixel. To stabilize these filters and the overall estimation, we insert a spatial regularization into the prediction lane. Through the recursive nature of the filter the regularization has the ability to perform filling-in of missing information over extended spatial extents. We benchmark our algorithm, which is implemented in the nVidia Cuda framework to exploit the processing power of modern graphical processing units (GPUs), against a state-of-the-art variational flow estimation algorithm that is also implemented in Cuda. The comparison shows that, while the variational method yields somewhat higher precision, our method is more than an order of magnitude faster and can thus operate in real-time on live video streams.
KeywordsOptical Flow Motion Vector Motion Estimation Coarse Scale Extended Algorithm
Unable to display preview. Download preview PDF.
- 1.Simoncelli, E.P., Jahne, B., Haussecker, H., Geissler, P.: Bayesian Multi-Scale Differential Optical Flow, vol. 2, pp. 397–422. Academic Press, San Diego (1999)Google Scholar
- 2.Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of Imaging Understanding Workshop, pp. 121–130 (1981)Google Scholar
- 4.Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–8 (October 2007)Google Scholar
- 5.Hauagge, D.C.: Homepage, http://www.liv.ic.unicamp.b/~hauagge/Daniel_Cabrini_Hauagge/Home_Page.html.
- 6.Rannacher, J.: Realtime 3d motion estimation on graphics hardware, Master’s thesis at Heidelberg University (2009)Google Scholar
- 7.Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 Optical Flow. In: Proceedings of British Machine Vision Conference (BMVC) (September 2009)Google Scholar
- 10.Chessa, M., Sabatini, S.P., Solari, F., Bisio, G.M.: A recursive approach to the design of adjustable linear models for complex motion analysis. In: Proceedings of the Fourth Conference on IASTED International Conference: Signal Processing, Pattern Recognition, and Applications, pp. 33–38. ACTA Press, Anaheim (2007)Google Scholar