Skip to main content
Book cover

European Conference on Computer Vision

ECCV 2020: Computer Vision – ECCV 2020 pp 36–53Cite as

Stereo Event-Based Particle Tracking Velocimetry for 3D Fluid Flow Reconstruction

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12374)

Abstract

Existing Particle Imaging Velocimetry techniques require the use of high-speed cameras to reconstruct time-resolved fluid flows. These cameras provide high-resolution images at high frame rates, which generates bandwidth and memory issues. By capturing only changes in the brightness with a very low latency and at low data rate, event-based cameras have the ability to tackle such issues. In this paper, we present a new framework that retrieves dense 3D measurements of the fluid velocity field using a pair of event-based cameras. First, we track particles inside the two event sequences in order to estimate their 2D velocity in the two sequences of images. A stereo-matching step is then performed to retrieve their 3D positions. These intermediate outputs are incorporated into an optimization framework that also includes physically plausible regularizers, in order to retrieve the 3D velocity field. Extensive experiments on both simulated and real data demonstrate the efficacy of our approach.

Keywords

  • Fluid imaging
  • Event-based camera
  • Particle Imaging Velocimetry
  • Stereo-PTV
  • Optimization

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-58526-6_3
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   109.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-58526-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   149.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.

Notes

  1. 1.

    The code is available on: https://github.com/vccimaging/StereoEventPTV.

References

  1. Adrian, R.J., Westerweel, J.: Particle Image Velocimetry. Cambridge University Press, Cambridge (2011)

    Google Scholar 

  2. Aguirre-Pablo, A.A., Aljedaani, A.B., Xiong, J., Idoughi, R., Heidrich, W., Thoroddsen, S.T.: Single-camera 3D PTV using particle intensities and structured light. Exp. Fluids 60(2), 1–13 (2019). https://doi.org/10.1007/s00348-018-2660-7

    CrossRef  Google Scholar 

  3. Álvarez, L., et al.: A new energy-based method for 3D motion estimation of incompressible PIV flows. Comput. Vis. Image Underst. 113(7), 802–810 (2009)

    CrossRef  Google Scholar 

  4. Atcheson, B., et al.: Time-resolved 3D capture of non-stationary gas flows. ACM Trans. Graph. 27(5), 132 (2008)

    CrossRef  Google Scholar 

  5. Bardow, P., Davison, A.J., Leutenegger, S.: Simultaneous optical flow and intensity estimation from an event camera. In: Proceedings of the CVPR, pp. 884–892 (2016)

    Google Scholar 

  6. Belden, J., Truscott, T.T., Axiak, M.C., Techet, A.H.: Three-dimensional synthetic aperture particle image velocimetry. Meas. Sci. Technol. 21(12), 125403 (2010)

    CrossRef  Google Scholar 

  7. Benosman, R., Clercq, C., Lagorce, X., Ieng, S.H., Bartolozzi, C.: Event-based visual flow. IEEE Trans. Neural Netw. Learn. Syst. 25(2), 407–417 (2013)

    CrossRef  Google Scholar 

  8. Berner, R., Brandli, C., Yang, M., Liu, S.C., Delbruck, T.: A 240 \(\times \) 180 10mW 12\(\upmu \)s latency sparse-output vision sensor for mobile applications. In: 2013 Symposium on VLSI Circuits, pp. C186–C187. IEEE (2013)

    Google Scholar 

  9. Berthelon, X., Chenegros, G., Libert, N., Sahel, J.A., Grieve, K., Benosman, R.: Full-field OCT technique for high speed event-based optical flow and particle tracking. Opt. Express 25(11), 12611–12621 (2017)

    CrossRef  Google Scholar 

  10. Biswas, S.: Schlieren image velocimetry (SIV). Physics of Turbulent Jet Ignition. ST, pp. 35–64. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76243-2_3

    CrossRef  Google Scholar 

  11. Borer, D., Delbruck, T., Rösgen, T.: Three-dimensional particle tracking velocimetry using dynamic vision sensors. Exp. Fluids 58(12), 1–7 (2017). https://doi.org/10.1007/s00348-017-2452-5

    CrossRef  Google Scholar 

  12. Brücker, C.: 3D scanning PIV applied to an air flow in a motored engine using digital high-speed video. Meas. Sci. Technol. 8(12), 1480 (1997)

    CrossRef  Google Scholar 

  13. Dabiri, D., Pecora, C.: Particle Tracking Velocimetry. IOP Publishing, Bristol (2020)

    Google Scholar 

  14. Delbruck, T., Lang, M.: Robotic goalie with 3 ms reaction time at 4% CPU load using event-based dynamic vision sensor. Front. Neurosci. 7, 223 (2013)

    CrossRef  Google Scholar 

  15. Drazen, D., Lichtsteiner, P., Häfliger, P., Delbrück, T., Jensen, A.: Toward real-time particle tracking using an event-based dynamic vision sensor. Exp. Fluids 51(5), 1465 (2011)

    CrossRef  Google Scholar 

  16. Eckert, M.L., Heidrich, W., Thürey, N.: Coupled fluid density and motion from single views. In: CGF, vol. 37, pp. 47–58. Wiley (2018)

    Google Scholar 

  17. Eckert, M.L., Um, K., Thuerey, N.: ScalarFlow: a large-scale volumetric data set of real-world scalar transport flows for computer animation and machine learning. ACM Trans. Graph. 38(6), 1–16 (2019)

    CrossRef  Google Scholar 

  18. Elsinga, G.E., Scarano, F., Wieneke, B., van Oudheusden, B.W.: Tomographic particle image velocimetry. Exp. Fluids 41(6), 933–947 (2006)

    CrossRef  Google Scholar 

  19. Fahringer, T.W., Lynch, K.P., Thurow, B.S.: Volumetric particle image velocimetry with a single plenoptic camera. Meas. Sci. Technol. 26(11), 115201 (2015)

    CrossRef  Google Scholar 

  20. Gallego, G., et al.: Event-based vision: a survey. arXiv preprint arXiv:1904.08405 (2019)

  21. Gesemann, S., Huhn, F., Schanz, D., Schröder, A.: From noisy particle tracks to velocity, acceleration and pressure fields using B-splines and penalties. In: 18th International Symposium on Applications of Laser and Imaging Techniques to Fluid Mechanics, Lisbon, Portugal, pp. 4–7 (2016)

    Google Scholar 

  22. Glover, A., Bartolozzi, C.: Event-driven ball detection and gaze fixation in clutter. In: Proceedings of the IROS, pp. 2203–2208. IEEE (2016)

    Google Scholar 

  23. Gregson, J., Ihrke, I., Thuerey, N., Heidrich, W.: From capture to simulation: connecting forward and inverse problems in fluids. ACM Trans. Graph. 33(4), 139 (2014)

    CrossRef  Google Scholar 

  24. Gregson, J., Krimerman, M., Hullin, M.B., Heidrich, W.: Stochastic tomography and its applications in 3D imaging of mixing fluids. ACM Trans. Graph. 31(4), 52:1–52:10 (2012)

    CrossRef  Google Scholar 

  25. Gu, J., Nayar, S.K., Grinspun, E., Belhumeur, P.N., Ramamoorthi, R.: Compressive structured light for recovering inhomogeneous participating media. IEEE Trans. PAMI 35(3), 1 (2012)

    Google Scholar 

  26. Hasinoff, S.W., Kutulakos, K.N.: Photo-consistent reconstruction of semitransparent scenes by density-sheet decomposition. IEEE Trans. PAMI 29(5), 870–885 (2007)

    CrossRef  Google Scholar 

  27. Hawkins, T., Einarsson, P., Debevec, P.: Acquisition of time-varying participating media. Technical report, University of Southern California Marina del Rey CA Institute for Creative (2005)

    Google Scholar 

  28. Heitz, D., Mémin, E., Schnörr, C.: Variational fluid flow measurements from image sequences: synopsis and perspectives. Exp. Fluids 48(3), 369–393 (2010). https://doi.org/10.1007/s00348-009-0778-3

    CrossRef  Google Scholar 

  29. Hinsch, K.D.: Holographic particle image velocimetry. Meas. Sci. Technol. 13(7), R61 (2002)

    CrossRef  Google Scholar 

  30. Hofstatter, M., Schön, P., Posch, C.: A SPARC-compatible general purpose address-event processor with 20-bit l0ns-resolution asynchronous sensor data interface in 0.18 \(\upmu \)m CMOS. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 4229–4232. IEEE (2010)

    Google Scholar 

  31. Hori, T., Sakakibara, J.: High-speed scanning stereoscopic PIV for 3D vorticity measurement in liquids. Meas. Sci. Technol. 15(6), 1067 (2004)

    CrossRef  Google Scholar 

  32. Ihrke, I., Goidluecke, B., Magnor, M.: Reconstructing the geometry of flowing water. In: Proceedings of the ICCV, vol. 2, pp. 1055–1060. IEEE (2005)

    Google Scholar 

  33. Ihrke, I., Magnor, M.: Image-based tomographic reconstruction of flames. In: Proceedings of the SCA, pp. 365–373 (2004)

    Google Scholar 

  34. Ji, Y., Ye, J., Yu, J.: Reconstructing gas flows using light-path approximation. In: Proceedings of the CVPR, pp. 2507–2514 (2013)

    Google Scholar 

  35. Jonassen, D.R., Settles, G.S., Tronosky, M.D.: Schlieren “PIV” for turbulent flows. Opt. Lasers Eng. 44(3–4), 190–207 (2006)

    CrossRef  Google Scholar 

  36. Knutsen, A.N., Lawson, J.M., Dawson, J.R., Worth, N.A.: A laser sheet self-calibration method for scanning PIV. Exp. Fluids 58(10), 1–13 (2017). https://doi.org/10.1007/s00348-017-2428-5

    CrossRef  Google Scholar 

  37. Lasinger, K., Vogel, C., Pock, T., Schindler, K.: 3D fluid flow estimation with integrated particle reconstruction. Int. J. Comput. Vis. 128(4), 1012–1027 (2020). https://doi.org/10.1007/s11263-019-01261-6

    CrossRef  MathSciNet  Google Scholar 

  38. Lasinger, K., Vogel, C., Schindler, K.: Volumetric flow estimation for incompressible fluids using the stationary stokes equations. In: Proceedings of the ICCV, pp. 2565–2573 (2017)

    Google Scholar 

  39. Maas, H., Gruen, A., Papantoniou, D.: Particle tracking velocimetry in three-dimensional flows. Exp. Fluids 15(2), 133–146 (1993). https://doi.org/10.1007/BF00223406

    CrossRef  Google Scholar 

  40. Machicoane, N., Aliseda, A., Volk, R., Bourgoin, M.: A simplified and versatile calibration method for multi-camera optical systems in 3D particle imaging. Rev. Sci. Instrum. 90(3), 035112 (2019)

    CrossRef  Google Scholar 

  41. Mahowald, M.: VLSI analogs of neuronal visual processing: a synthesis of form and function. Ph.D. thesis, California Institute of Technology Pasadena (1992)

    Google Scholar 

  42. Mei, D., Ding, J., Shi, S., New, T.H., Soria, J.: High resolution volumetric dual-camera light-field PIV. Exp. Fluids 60(8), 1–21 (2019). https://doi.org/10.1007/s00348-019-2781-7

    CrossRef  Google Scholar 

  43. Meng, H., Hussain, F.: Holographic particle velocimetry: a 3D measurement technique for vortex interactions, coherent structures and turbulence. Fluid Dyn. Res. 8(1–4), 33 (1991)

    CrossRef  Google Scholar 

  44. Morris, N.J., Kutulakos, K.N.: Dynamic refraction stereo. IEEE Trans. Pattern Anal. Mach. Intell. 33(8), 1518–1531 (2011)

    CrossRef  Google Scholar 

  45. Ni, Z., Pacoret, C., Benosman, R., Ieng, S., Régnier, S.: Asynchronous event-based high speed vision for microparticle tracking. J. Microsc. 245(3), 236–244 (2012)

    CrossRef  Google Scholar 

  46. Okamoto, K., Nishio, S., Saga, T., Kobayashi, T.: Standard images for particle-image velocimetry. Meas. Sci. Technol. 11(6), 685 (2000)

    CrossRef  Google Scholar 

  47. Pereira, F., Gharib, M., Dabiri, D., Modarress, D.: Defocusing digital particle image velocimetry: a 3-component 3-dimensional DPIV measurement technique. Application to bubbly flows. Exp. Fluids 29(1), S078–S084 (2000). https://doi.org/10.1007/s003480070010

    CrossRef  Google Scholar 

  48. Pereira, F., Gharib, M.: Defocusing digital particle image velocimetry and the three-dimensional characterization of two-phase flows. Meas. Sci. Technol. 13(5), 683 (2002)

    CrossRef  Google Scholar 

  49. Posch, C., Matolin, D., Wohlgenannt, R.: A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid-State Circuits 46(1), 259–275 (2010)

    CrossRef  Google Scholar 

  50. Prasad, A.K.: Stereoscopic particle image velocimetry. Exp. Fluids 29(2), 103–116 (2000)

    CrossRef  Google Scholar 

  51. Raffel, M.: Background-oriented schlieren (BOS) techniques. Exp. Fluids 56(3), 1–17 (2015). https://doi.org/10.1007/s00348-015-1927-5

    CrossRef  Google Scholar 

  52. Raffel, M., Willert, C.E., Scarano, F., Kähler, C.J., Wereley, S.T., Kompenhans, J.: Particle Image Velocimetry. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-68852-7

    CrossRef  Google Scholar 

  53. Rebecq, H., Gallego, G., Mueggler, E., Scaramuzza, D.: EMVS: event-based multi-view stereo-3D reconstruction with an event camera in real-time. Int. J. Comput. Vis. 126(12), 1394–1414 (2018)

    CrossRef  Google Scholar 

  54. Rebecq, H., Gehrig, D., Scaramuzza, D.: ESIM: an open event camera simulator. In: Conference on Robot Learning, pp. 969–982 (2018)

    Google Scholar 

  55. Rebecq, H., Ranftl, R., Koltun, V., Scaramuzza, D.: High speed and high dynamic range video with an event camera. IEEE Trans. Pattern Anal. Mach. Intell. (2019)

    Google Scholar 

  56. Richard, H., Raffel, M.: Principle and applications of the background oriented schlieren (BOS) method. Meas. Sci. Technol. 12(9), 1576 (2001)

    CrossRef  Google Scholar 

  57. Rogister, P., Benosman, R., Ieng, S.H., Lichtsteiner, P., Delbruck, T.: Asynchronous event-based binocular stereo matching. IEEE Trans. Neural Netw. Learn. Syst. 23(2), 347–353 (2011)

    CrossRef  Google Scholar 

  58. Ruhnau, P., Guetter, C., Putze, T., Schnörr, C.: A variational approach for particle tracking velocimetry. Meas. Sci. Technol. 16(7), 1449 (2005)

    CrossRef  Google Scholar 

  59. Ruhnau, P., Schnörr, C.: Optical stokes flow estimation: an imaging-based control approach. Exp. Fluids 42(1), 61–78 (2007). https://doi.org/10.1007/s00348-006-0220-z

    CrossRef  Google Scholar 

  60. Ruhnau, P., Stahl, A., Schnörr, C.: On-line variational estimation of dynamical fluid flows with physics-based spatio-temporal regularization. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 444–454. Springer, Heidelberg (2006). https://doi.org/10.1007/11861898_45

    CrossRef  Google Scholar 

  61. Scarano, F.: Tomographic PIV: principles and practice. Meas. Sci. Technol. 24(1), 012001 (2012)

    CrossRef  Google Scholar 

  62. Schanz, D., Gesemann, S., Schröder, A.: Shake-The-Box: Lagrangian particle tracking at high particle image densities. Exp. Fluids 57(5), 1–27 (2016). https://doi.org/10.1007/s00348-016-2157-1

    CrossRef  Google Scholar 

  63. Schneiders, J.F.G., Scarano, F.: Dense velocity reconstruction from tomographic PTV with material derivatives. Exp. Fluids 57(9), 1–22 (2016). https://doi.org/10.1007/s00348-016-2225-6

    CrossRef  Google Scholar 

  64. Serrano-Gotarredona, T., Linares-Barranco, B.: A 128 \(\times \) 128 1.5% contrast sensitivity 0.9% FPN 3 \(\upmu \)s latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers. IEEE J. Solid-State Circuits 48(3), 827–838 (2013)

    Google Scholar 

  65. Stam, J.: Stable fluids. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, pp. 121–128 (1999)

    Google Scholar 

  66. Tan, Z.P., Thurow, B.S.: Time-resolved 3D flow-measurement with a single plenoptic-camera. In: AIAA Scitech 2019 Forum, p. 0267 (2019)

    Google Scholar 

  67. Tropea, C., Yarin, A.L.: Springer Handbook of Experimental Fluid Mechanics. SHB. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-30299-5

    CrossRef  Google Scholar 

  68. Wang, H., Liao, M., Zhang, Q., Yang, R., Turk, G.: Physically guided liquid surface modeling from videos. ACM Trans. Graph. (TOG) 28(3), 1–11 (2009)

    Google Scholar 

  69. Xiong, J., Fu, Q., Idoughi, R., Heidrich, W.: Reconfigurable rainbow PIV for 3D flow measurement. In: Proceedings of the ICCP, pp. 1–9. IEEE (2018)

    Google Scholar 

  70. Xiong, J., et al.: Rainbow particle imaging velocimetry for dense 3D fluid velocity imaging. ACM Trans. Graph. 36(4), 36 (2017)

    CrossRef  Google Scholar 

  71. Yoon, S.Y., Kim, K.C.: 3D particle position and 3D velocity field measurement in a microvolume via the defocusing concept. Meas. Sci. Technol. 17(11), 2897 (2006)

    CrossRef  Google Scholar 

  72. Zang, G., et al.: TomoFluid: reconstructing dynamic fluid from sparse view videos. In: Proceedings of the CVPR, pp. 1870–1879 (2020)

    Google Scholar 

  73. Zhu, A.Z., Atanasov, N., Daniilidis, K.: Event-based feature tracking with probabilistic data association. In: 2017 IEEE International Conference on Robotics and Automation (ICRA) pp. 4465–4470. IEEE (2017)

    Google Scholar 

  74. Zhu, A.Z., Yuan, L., Chaney, K., Daniilidis, K.: EV-FlowNet: self-supervised optical flow estimation for event-based cameras. arXiv preprint arXiv:1802.06898 (2018)

  75. Zihao Zhu, A., Atanasov, N., Daniilidis, K.: Event-based visual inertial odometry. In: Proceedings of the CVPR, pp. 5391–5399 (2017)

    Google Scholar 

Download references

Acknowledgments

This work was supported by King Abdullah University of Science and Technology as part of VCC Center Competitive Funding. The authors would like to thank the anonymous reviewers for their valuable comments. We thank Hadi Amata for his help in the design of the hexagonal tank and the camera extension tubes. We also thank Congli Wang for helping in the use of the event cameras.

Author information

Authors and Affiliations

  1. King Abdullah University of Science and Technology, Thuwal, Saudi Arabia

    Yuanhao Wang, Ramzi Idoughi & Wolfgang Heidrich

Authors
  1. Yuanhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramzi Idoughi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wolfgang Heidrich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanhao Wang .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Prof. Dr. Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 2 (mp4 86158 KB)

Supplementary material 1 (pdf 11150 KB)

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Wang, Y., Idoughi, R., Heidrich, W. (2020). Stereo Event-Based Particle Tracking Velocimetry for 3D Fluid Flow Reconstruction. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12374. Springer, Cham. https://doi.org/10.1007/978-3-030-58526-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58526-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58525-9

  • Online ISBN: 978-3-030-58526-6

  • eBook Packages: Computer ScienceComputer Science (R0)