Advertisement

On the Nonlinear Statistics of Optical Flow

  • Henry AdamsEmail author
  • Johnathan Bush
  • Brittany Carr
  • Lara Kassab
  • Joshua Mirth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11382)

Abstract

In A naturalistic open source movie for optical flow evaluation, Butler et al. create a database of ground-truth optical flow from the computer-generated video Sintel. We study the high-contrast \(3\times 3\) patches from this video, and provide evidence that this dataset is well-modeled by a torus (a nonlinear 2-dimensional manifold). Our main tools are persistent homology and zigzag persistence, which are popular techniques from the field of computational topology. We show that the optical flow torus model is naturally equipped with the structure of a fiber bundle, which is furthermore related to the statistics of range images.

Keywords

Optical flow Computational topology Persistent homology Fiber bundle Zigzag persistence 

Notes

Acknowledgements

We would like to thank Gunnar Carlsson, Bradley Nelson, Jose Perea, and Guillermo Sapiro for helpful conversations.

References

  1. 1.
    Adams, H., Atanasov, A., Carlsson, G.: Nudged elastic band in topological data analysis. Topological Methods Nonlinear Anal. 45(1), 247–272 (2015)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Adams, H., Carlsson, G.: On the nonlinear statistics of range image patches. SIAM J. Imaging Sci. 2(1), 110–117 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Adams, H., et al.: Persistence images: a vector representation of persistent homology. J. Mach. Learn. Res. 18(8), 1–35 (2017)MathSciNetGoogle Scholar
  4. 4.
    Armstrong, M.A.: Basic Topology. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-1-4757-1793-8zbMATHCrossRefGoogle Scholar
  5. 5.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)CrossRefGoogle Scholar
  6. 6.
    Bao, W., Li, H., Li, N., Jiang, W.: A liveness detection method for face recognition based on optical flow field. In: 2009 International Conference on Image Analysis and Signal Processing, IASP 2009, pp. 233–236. IEEE (2009)Google Scholar
  7. 7.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)CrossRefGoogle Scholar
  8. 8.
    Baryshnikov, Y., Ghrist, R.: Target enumeration via euler characteristic integrals. SIAM J. Appl. Math. 70(3), 825–844 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Bauer, U.: Ripser: a lean C++ code for the computation of Vietoris-Rips persistence barcodes. Software (2017). https://github.com/Ripser/ripser
  10. 10.
    Beauchemin, S.S., Barron, J.L.: The computation of optical flow. ACM Comput. Surv. (CSUR) 27(3), 433–466 (1995)CrossRefGoogle Scholar
  11. 11.
    Bendich, P., Marron, J.S., Miller, E., Pieloch, A., Skwerer, S.: Persistent homology analysis of brain artery trees. Ann. Appl. Stat. 10(1), 198 (2016)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Bubenik, P.: Statistical topological data analysis using persistence landscapes. J. Mach. Learn. Res. 16(1), 77–102 (2015)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Burghelea, D., Dey, T.K.: Topological persistence for circle-valued maps. Discrete Comput. Geom. 50(1), 69–98 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33783-3_44CrossRefGoogle Scholar
  15. 15.
    Carlsson, G.: Topology and data. Bull. Am. Math. Soc. 46(2), 255–308 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Carlsson, G., De Silva, V., Morozov, D.: Zigzag persistent homologyand real-valued functions. In: Proceedings of the Twenty-Fifth annual Symposium on Computational Geometry, pp. 247–256. ACM (2009)Google Scholar
  17. 17.
    Carlsson, G., Ishkhanov, T., De Silva, V., Zomorodian, A.: On the local behavior of spaces of natural images. Int. J. Comput. Vis. 76(1), 1–12 (2008)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Carlsson, G., de Silva, V.: Zigzag persistence. Found. Comput. Math. 10(4), 367–405 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Chung, M.K., Bubenik, P., Kim, P.T.: Persistence diagrams of cortical surface data. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 386–397. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-02498-6_32CrossRefGoogle Scholar
  20. 20.
    De Silva, V., Carlsson, G.: Topological estimation using witness complexes. SPBG 4, 157–166 (2004)Google Scholar
  21. 21.
    Edelsbrunner, H., Harer, J.L.: Computational Topology: An Introduction. American Mathematical Society, Providence (2010)zbMATHGoogle Scholar
  22. 22.
    Edelsbrunner, H., Letscher, D., Zomorodian, A.: Topological persistence and simplification. In: 2000 Proceedings of 41st Annual Symposium on Foundations of Computer Science, pp. 454–463. IEEE (2000)Google Scholar
  23. 23.
    Fleet, D., Weiss, Y.: Optical flow estimation. In: Paragios, N., Chen, Y., Faugeras, O. (eds.) Handbook of Mathematical Models in Computer Vision, pp. 237–257. Springer, Boston (2006).  https://doi.org/10.1007/0-387-28831-7_15CrossRefGoogle Scholar
  24. 24.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. (IJRR) 32, 1231–1237 (2013)CrossRefGoogle Scholar
  25. 25.
    Hatcher, A.: Algebraic Topology. Cambridge University Press, Cambridge (2002)zbMATHGoogle Scholar
  26. 26.
    Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)CrossRefGoogle Scholar
  27. 27.
    Huang, J., Lee, A.B., Mumford, D.B.: Statistics of range images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 324–332 (2000)Google Scholar
  28. 28.
    Lee, A.B., Pedersen, K.S., Mumford, D.: The nonlinear statistics of high-contrast patches in natural images. Int. J. Comput. Vis. 54(1–3), 83–103 (2003)zbMATHCrossRefGoogle Scholar
  29. 29.
    Lum, P., et al.: Extracting insights from the shape of complex data using topology. Sci. Rep. 3, 1236 (2013)CrossRefGoogle Scholar
  30. 30.
    Mac Aodha, O., Humayun, A., Pollefeys, M., Brostow, G.J.: Learning a confidence measure for optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1107–1120 (2013)CrossRefGoogle Scholar
  31. 31.
  32. 32.
    Roosendaal, T.: Sintel. Blender Foundation, Durian Open Movie Project (2010). http://www.sintel.org/
  33. 33.
    Roth, S., Black, M.J.: On the spatial statistics of optical flow. Int. J. Comput. Vis. 74(1), 33–50 (2007)CrossRefGoogle Scholar
  34. 34.
    de Silva, V., Ghrist, R.: Coordinate-free coverage in sensor networks with controlled boundaries via homology. Int. J. Robot. Res. 25(12), 1205–1222 (2006)zbMATHCrossRefGoogle Scholar
  35. 35.
    Topaz, C.M., Ziegelmeier, L., Halverson, T.: Topological data analysis of biological aggregation models. PloS One 10(5), e0126383 (2015)CrossRefGoogle Scholar
  36. 36.
    Xia, K., Wei, G.W.: Persistent homology analysis of protein structure, flexibility, and folding. Int. J. Numer. Methods Biomed. Eng. 30(8), 814–844 (2014)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Zomorodian, A., Carlsson, G.: Computing persistent homology. Discrete Comput. Geom. 33(2), 249–274 (2005)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Henry Adams
    • 1
    Email author
  • Johnathan Bush
    • 1
  • Brittany Carr
    • 1
  • Lara Kassab
    • 1
  • Joshua Mirth
    • 1
  1. 1.Colorado State UniversityFort CollinsUSA

Personalised recommendations