Abstract
We present principled algorithms for measuring the velocity and 3D location of refractive fluids, such as hot air or gas, from natural videos with textured backgrounds. Our main observation is that intensity variations related to movements of refractive fluid elements, as observed by one or more video cameras, are consistent over small space-time volumes. We call these intensity variations “refraction wiggles”, and use them as features for tracking and stereo fusion to recover the fluid motion and depth from video sequences. We give algorithms for 1) measuring the (2D, projected) motion of refractive fluids in monocular videos, and 2) recovering the 3D position of points on the fluid from stereo cameras. Unlike pixel intensities, wiggles can be extremely subtle and cannot be known with the same level of confidence for all pixels, depending on factors such as background texture and physical properties of the fluid. We thus carefully model uncertainty in our algorithms for robust estimation of fluid motion and depth. We show results on controlled sequences, synthetic simulations, and natural videos. Different from previous approaches for measuring refractive flow, our methods operate directly on videos captured with ordinary cameras, do not require auxiliary sensors, light sources or designed backgrounds, and can correctly detect the motion and location of refractive fluids even when they are invisible to the naked eye.
Chapter PDF
Similar content being viewed by others
Keywords
- Particle Image Velocimetry
- Camera Center
- Brightness Constancy
- Background Orient Schlieren
- Representative Frame
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Adrian, R.J., Westerweel, J.: Particle image velocimetry. Cambridge University Press (2011)
Alterman, M., Schechner, Y.Y., Perona, P., Shamir, J.: Detecting motion through dynamic refraction. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013)
Alterman, M., Schechner, Y.Y., Swirski, Y.: Triangulation in random refractive distortions. In: 2013 IEEE International Conference on Computational Photography (ICCP), pp. 1–10. IEEE (2013)
Alterman, M., Schechner, Y.Y., Vo, M., Narasimhan, S.G.: Passive tomography of turbulence strength. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part IV. LNCS, vol. 8692, pp. 47–60. Springer, Heidelberg (2014)
Arnaud, E., Mémin, É., Sosa, R., Artana, G.: A fluid motion estimator for schlieren image velocimetry. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 198–210. Springer, Heidelberg (2006)
Atcheson, B., Ihrke, I., Heidrich, W., Tevs, A., Bradley, D., Magnor, M., Seidel, H.P.: Time-resolved 3D capture of non-stationary gas flows. ACM Trans. Graph. 27(5), 132:1–132:9 (2008)
Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. International Journal of Computer Vision 92(1), 1–31 (2011)
Dalziel, S.B., Hughes, G.O., Sutherland, B.R.: Whole-field density measurements by “synthetic schlieren”. Experiments in Fluids 28(4), 322–335 (2000)
Ding, Y., Li, F., Ji, Y., Yu, J.: Dynamic fluid surface acquisition using a camera array. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2478–2485 (2011)
Elsinga, G., van Oudheusden, B., Scarano, F., Watt, D.: Assessment and application of quantitative schlieren methods: Calibrated color schlieren and background oriented schlieren. Experiments in Fluids 36(2), 309–325 (2004)
Fleet, D., Weiss, Y.: Optical flow estimation. In: Handbook of Mathematical Models in Computer Vision, pp. 237–257. Springer, US (2006)
Hargather, M.J., Settles, G.S.: Natural-background-oriented schlieren imaging. Experiments in Fluids 48(1), 59–68 (2010)
Has, P., Herzet, C., Mmin, E., Heitz, D., Mininni, P.D.: Bayesian estimation of turbulent motion. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1343–1356 (2013)
Jonassen, D.R., Settles, G.S., Tronosky, M.D.: Schlieren “PIV” for turbulent flows. Optics and Lasers in Engineering 44(3-4), 190–207 (2006)
Meier, G.: Computerized background-oriented schlieren. Experiments in Fluids 33(1), 181–187 (2002)
Morris, N.J., Kutulakos, K.N.: Dynamic refraction stereo. In: 2005 IEEE International Conference on Computer Vision (ICCV), vol. 2, pp. 1573–1580 (2005)
Raffel, M., Tung, C., Richard, H., Yu, Y., Meier, G.: Background oriented stereoscopic schlieren (BOSS) for full scale helicopter vortex characterization. In: Proc. of International Symposium on Flow Visualization (2000)
Richard, H., Raffel, M.: Principle and applications of the background oriented schlieren (BOS) method. Measurement Science and Technology 12(9), 1576 (2001)
Ruhnau, P., Kohlberger, T., Schnörr, C., Nobach, H.: Variational optical flow estimation for particle image velocimetry. Experiments in Fluids 38(1), 21–32 (2005)
Ruhnau, P., Schnörr, C.: Optical stokes flow estimation: An imaging-based control approach. Experiments in Fluids 42(1), 61–78 (2007)
Ruhnau, P., Stahl, A., Schnörr, C.: On-line variational estimation of dynamical fluid flows with physics-based spatio-temporal regularization. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) DAGM 2006. LNCS, vol. 4174, pp. 444–454. Springer, Heidelberg (2006)
Schardin, H.: Die schlierenverfahren und ihre anwendungen. In: Ergebnisse der Exakten Naturwissenschaften, pp. 303–439. Springer (1942)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47(1-3), 7–42 (2002)
Settles, G.S.: Schlieren and shadowgraph techniques: visualizing phenomena in transparent media, vol. 2. Springer, Berlin (2001)
Settles, G.S.: The penn state full-scale schlieren system. In: Proc. of International Symposium on Flow Visualization (2004)
Stam, J.: Stable fluids. In: Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1999, pp. 121–128. ACM Press/Addison-Wesley Publishing Co., New York (1999)
Tian, Y., Narasimhan, S., Vannevel, A.: Depth from optical turbulence. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 246–253. IEEE (June 2012)
Wetzstein, G., Raskar, R., Heidrich, W.: Hand-held schlieren photography with light field probes. In: 2011 IEEE International Conference on Computational Photography (ICCP), pp. 1–8 (April 2011)
Wetzstein, G., Roodnick, D., Heidrich, W., Raskar, R.: Refractive shape from light field distortion. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1180–1186 (2011)
Ye, J., Ji, Y., Li, F., Yu, J.: Angular domain reconstruction of dynamic 3d fluid surfaces. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 310–317 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Xue, T., Rubinstein, M., Wadhwa, N., Levin, A., Durand, F., Freeman, W.T. (2014). Refraction Wiggles for Measuring Fluid Depth and Velocity from Video. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8691. Springer, Cham. https://doi.org/10.1007/978-3-319-10578-9_50
Download citation
DOI: https://doi.org/10.1007/978-3-319-10578-9_50
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10577-2
Online ISBN: 978-3-319-10578-9
eBook Packages: Computer ScienceComputer Science (R0)