Monocular Epipolar Constraint for Optical Flow Estimation

  • Mahmoud A. Mohamed
  • M. Hossein Mirabdollah
  • Bärbel Mertsching
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10528)

Abstract

In this paper, the usage of the monocular epipolar geometry for the calculation of optical flow is investigated. We derive the necessary formulation to use the epipolar constraint for the calculation of differential optical flow using the total variational model in a multi-resolution pyramid scheme. Therefore, we minimize an objective function which contains the epipolar constraint with a residual function based on different types of descriptors (brightness, HOG, CENSUS and MLDP). For the calculation of epipolar lines, the relevant fundamental matrices are calculated based on the 7- and 8- point methods. Moreover, SIFT and Lukas-Kanade methods are used to obtain matched features between two consecutive frames, by which fundamental matrices can be calculated. The effect of using different combination of the feature matching methods, fundamental matrix calculation and descriptors are evaluated based on the KITTI 2012 dataset.

References

  1. 1.
    Bernd, K., Henning, L.: Trinocular optical flow estimation for intelligent vehicle applications. In: 15th International IEEE Conference on Intelligent Transportation Systems (ITSC). IEEE, September 2012Google Scholar
  2. 2.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24673-2_3 CrossRefGoogle Scholar
  3. 3.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. (IJRR) 32, 1231–1237 (2013)CrossRefGoogle Scholar
  4. 4.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2003)MATHGoogle Scholar
  5. 5.
    Kennedy, R., Taylor, C.J.: Optical flow with geometric occlusion estimation and fusion of multiple frames. In: Tai, X.-C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. LNCS, vol. 8932, pp. 364–377. Springer, Cham (2015). doi:10.1007/978-3-319-14612-6_27 Google Scholar
  6. 6.
    Mohamed, M., Mertsching, B.: TV-L1 optical flow estimation with image details recovering based on modified census transform. Adv. Vis. Comput. 7431, 482–491 (2012)Google Scholar
  7. 7.
    Mohamed, M.A., Mirabdollah, M.H., Mertsching, B.: Differential optical flow estimation under monocular epipolar line constraint. In: Nalpantidis, L., Krüger, V., Eklundh, J.-O., Gasteratos, A. (eds.) ICVS 2015. LNCS, vol. 9163, pp. 354–363. Springer, Cham (2015). doi:10.1007/978-3-319-20904-3_32 CrossRefGoogle Scholar
  8. 8.
    Mohamed, M.A., Boddeker, C., Mertsching, B.: Real-time moving objects tracking for mobile-robots using motion information. In: 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 1–6. IEEE (2014)Google Scholar
  9. 9.
    Mohamed, M.A., Rashwan, H.A., Mertsching, B., Garcia, M.A., Puig, D.: On improving the robustness of variational optical flow against illumination changes. In: Proceedings of the 4th ACM/IEEE International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Stream, pp. 1–8. ACM (2013)Google Scholar
  10. 10.
    Mohamed, M.A., Rashwan, H.A., Mertsching, B., Garcia, M.A., Puig, D.: Illumination-robust optical flow using a local directional pattern. IEEE Trans. Circuits Syst. Video Technol. 24, 1–9 (2014)CrossRefGoogle Scholar
  11. 11.
    Müller, T., Rabe, C., Rannacher, J., Franke, U., Mester, R.: Illumination-robust dense optical flow using census signatures. In: Mester, R., Felsberg, M. (eds.) DAGM 2011. LNCS, vol. 6835, pp. 236–245. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23123-0_24 CrossRefGoogle Scholar
  12. 12.
    Ranftl, R., Vineet, V., Chen, Q., Koltun, V.: Dense monocular depth estimation in complex dynamic scenes. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4058–4066, June 2016Google Scholar
  13. 13.
    Rashwan, H.A., Mohamed, M.A., García, M.A., Mertsching, B., Puig, D.: Illumination robust optical flow model based on histogram of oriented gradients. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 354–363. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40602-7_38 CrossRefGoogle Scholar
  14. 14.
    Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: Epicflow: edge-preserving interpolation of correspondences for optical flow. In: The IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7–12, June, pp. 1164–1172. IEEE (2015)Google Scholar
  15. 15.
    Taniai, T., Sinha, S., Sato, Y.: Fast multi-frame stereo scene flow with motion segmentation. In: The IEEE Computer Vision and Pattern Recognition (CVPR). IEEE, March 2017Google Scholar
  16. 16.
    Valgaerts, L., Bruhn, A., Weickert, J.: A variational model for the joint recovery of the fundamental matrix and the optical flow. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 314–324. Springer, Heidelberg (2008). doi:10.1007/978-3-540-69321-5_32 CrossRefGoogle Scholar
  17. 17.
    Yamaguchi, K., McAllester, D.A., Urtasun, R.: Robust monocular epipolar flow estimation. In: The IEEE Computer Vision and Pattern Recognition (CVPR), pp. 1862–1869. IEEE (2013)Google Scholar
  18. 18.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74936-3_22 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mahmoud A. Mohamed
    • 1
  • M. Hossein Mirabdollah
    • 1
  • Bärbel Mertsching
    • 1
  1. 1.GET LabUniversity of PaderbornPaderbornGermany

Personalised recommendations