Abstract
In variational optical flow estimation, second order regularisation plays an important role, since it offers advantages in the context of non-fronto-parallel motion. However, in contrast to first order smoothness constraints, most second order regularisers are limited to isotropic concepts. Moreover, the few existing anisotropic concepts are lacking a comparison so far. Hence, our contribution is twofold. (i) First, we juxtapose general concepts for isotropic and anisotropic second order regularization based on direct second order methods, infimal convolution techniques, and indirect coupling models. For all the aforementioned strategies suitable optical flow regularisers are derived. (ii) Second, we show that modelling anisotropic second order smoothness terms gives an additional degree of freedom when penalising deviations from smoothness. This in turn allows us to propose a novel anisotropic strategy which we call double anisotropic regularisation. Experiments on the two KITTI benchmarks show the qualitative differences between the different strategies. Moreover, they demonstrate that the novel concept of double anisotropic regularisation is able to produce excellent results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Braux-Zin, J., Dupont, R., Bartoli, A.: A general dense image matching framework combining direct and feature-based costs. In: Proceedings of International Conference on Computer Vision, pp. 185–192 (2013)
Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imaging Sci. 3(3), 492–526 (2010)
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24673-2_3
Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)
Chan, T., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM J. Sci. Comput. 22(2), 503–516 (2000)
Charbonnier, P., Blanc-Féraud, L., Aubert, G., Barlaud, M.: Deterministic edge-preserving regularization in computed imaging. IEEE Trans. Image Process. 6(2), 298–311 (1997)
Demetz, O., Stoll, M., Volz, S., Weickert, J., Bruhn, A.: Learning brightness transfer functions for the joint recovery of illumination changes and optical flow. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 455–471. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_30
Ferstl, D., Reinbacher, C., Ranftl, R., Rüther, M., Bischof, H.: Image guided depth upsampling using anisotropic total generalized variation. In: Proceedings of International Conference on Computer Vision, pp. 993–1000 (2013)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361 (2012)
Hafner, D., Schroers, C., Weickert, J.: Introducing maximal anisotropy into second order coupling models. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 79–90. Springer, Cham (2015). doi:10.1007/978-3-319-24947-6_7
Hewer, A., Weickert, J., Scheffer, T., Seibert, H., Diebels, S.: Lagrangian strain tensor computation with higher order variational models. In: Proceedings of British Machine Vision Conference, pp. 129.1–129.10 (2013)
Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)
Lefkimmiatis, S., Ward, J.P., Unser, M.: Hessian Schatten-norm regularization for linear inverse problems. IEEE Trans. Image Process. 22(5), 1873–1888 (2013)
Lellmann, J., Morel, J.-M., Schönlieb, C.-B.: Anisotropic third-order regularization for sparse digital elevation models. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 161–173. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38267-3_14
Lenzen, F., Becker, F., Lellmann, J.: Adaptive second-order total variation: an approach aware of slope discontinuities. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 61–73. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38267-3_6
Lysaker, M., Lundervold, A., Tai, X.C.: Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Process. 12(12), 1579–1590 (2003)
Menze, M., Geiger, A.: Object scene flow for autonomous vehicles. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3061–3070 (2015)
Nagel, H.-H., Enkelmann, W.: An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Patt. Anal. Mach. Intell. 8(5), 565–593 (1986)
Nir, T., Bruckstein, A.M., Kimmel, R.: Over-parameterized variational optical flow. Int. J. Comput. Vis. 76(2), 205–216 (2008)
Perona, P., Malik, J.: Scale space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 2, 629–639 (1990)
Ranftl, R., Gehrig, S., Pock, T., Bischof, H.: Pushing the limits of stereo using variational stereo estimation. In: IEEE Intelligent Vehicles Symposium, pp. 401–407 (2012)
Ranftl, R., Bredies, K., Pock, T.: Non-local total generalized variation for optical flow estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 439–454. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_29
Ranftl, R.: Higher-Order Variational Methods for Dense Correspondence Problems. Ph.D thesis. Graz University of Technology, Austria (2014)
Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: Edge-preserving interpolation of correspondences for optical flow. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1164–1172 (2015)
Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Scherzer, O.: Denoising with higher order derivatives of bounded variation and an application to parameter estimation. Computing 60(1), 1–27 (1998)
Sun, D., Roth, S., Lewis, J.P., Black, M.J.: Learning optical flow. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 83–97. Springer, Heidelberg (2008). doi:10.1007/978-3-540-88690-7_7
Trobin, W., Pock, T., Cremers, D., Bischof, H.: An Unbiased second-order prior for high-accuracy motion estimation. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 396–405. Springer, Heidelberg (2008). doi:10.1007/978-3-540-69321-5_40
Vogel, O., Bruhn, A., Weickert, J., Didas, S.: Direct shape-from-shading with adaptive higher order regularisation. In: Sgallari, F., Murli, A., Paragios, N. (eds.) SSVM 2007. LNCS, vol. 4485, pp. 871–882. Springer, Heidelberg (2007). doi:10.1007/978-3-540-72823-8_75
Weickert, J., Welk, M., Wickert, M.: L 2-Stable nonstandard finite differences for anisotropic diffusion. In: Kuijper, A., Bredies, K., Pock, T., Bischof, H. (eds.) SSVM 2013. LNCS, vol. 7893, pp. 380–391. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38267-3_32
Weickert, J., Schnörr, C.: A theoretical framework for convex regularizers in PDE-based computation of image motion. Int. J. Comput. Vision 45(3), 245–264 (2001)
Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-\(L^1\) optic flow. In: Proceedings of British Machine Vision Conference (2009)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-\(L^1\) optical flow. In: Proceedings of German Pattern Recognition, pp. 214–223 (2007)
Zimmer, H., Bruhn, A., Weickert, J.: Optic flow in harmony. Int. J. Comput. Vis. 93(3), 368–388 (2011)
Acknowledgements
We thank the German Research Foundation (DFG) for financial support within project B04 of SFB/Transregio 161.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Maurer, D., Stoll, M., Volz, S., Gairing, P., Bruhn, A. (2017). A Comparison of Isotropic and Anisotropic Second Order Regularisers for Optical Flow. In: Lauze, F., Dong, Y., Dahl, A. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2017. Lecture Notes in Computer Science(), vol 10302. Springer, Cham. https://doi.org/10.1007/978-3-319-58771-4_43
Download citation
DOI: https://doi.org/10.1007/978-3-319-58771-4_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58770-7
Online ISBN: 978-3-319-58771-4
eBook Packages: Computer ScienceComputer Science (R0)