# Super-resolution of turbulent passive scalar images using data assimilation

## Abstract

In this paper, the problem of improving the quality of low-resolution passive scalar image sequences is addressed. This situation, known as “image super-resolution” in computer vision, aroused to our knowledge very few applications in the field of fluid visualization. Yet, in most image acquisition devices, the spatial resolution of the acquired data is limited by the sensor physical properties, while users often require higher-resolution images for further processing and analysis of the system of interest. The originality of the approach presented in this paper is to link the image super-resolution process together with the large eddy simulation framework in order to derive a complete super-resolution technique. We first start by defining two categories of fine-scale components we aim to reconstruct. Then, using a deconvolution procedure as well as data assimilation tools, we show how to partially recover some of these missing components within the low-resolution images while ensuring the temporal consistency of the solution. This method is evaluated using both synthetic and real image data. Finally, we demonstrate how the produced high-resolution images can improve a posteriori analysis such as motion field estimation.

## Keywords

Scalar Field Direct Numerical Simulation Motion Field Average Angular Error Image Acquisition Device## Notes

### Acknowledgments

This work is supported by CaiYuanPei EGIDE grant.

## References

- Baker S, Kanade T (2002) Limits on super-resolution and how to break them? IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183CrossRefGoogle Scholar
- Bennett AF (1992) Inverse methods in physical oceanography. Cambridge monographs on mechanics and applied mathematics. Cambridge University Press, Cambridge, New YorkCrossRefGoogle Scholar
- Borman S, Stevenson RL (1998) Super-resolution from image sequences—a review. In: Proceedings of the 1998 midwest symposium on circuits and systemsGoogle Scholar
- Bruhn A, Weickert J, Schnörr C (2005) Lucas/kanade meets horn/schunck: combining local and global optic flow methods. Int J Comput Vis 61(3):211–231CrossRefGoogle Scholar
- Cassisa C, Simoens S, Prinet V, Shao L (2011) Subgrid scale formulation of optical flow for the study of turbulent flow. Exp Fluids 51(6):1739–1754CrossRefGoogle Scholar
- Chen X, Zillé P, Shao L, Corpetti T (2015) Optical flow for incompressible turbulence motion estimation. Exp Fluids 56(1):1–14CrossRefGoogle Scholar
- Corpetti T, Memin E, Perez P (2002) Dense estimation of fluid flows. IEEE Trans Pattern Anal Mach Intell 24(3):365–380CrossRefGoogle Scholar
- Corpetti T, Mémin E, Pérez P (2003) Extraction of singular points from dense motion fields: an analytic approach. J Math Imaging Vis 19(3):175–198CrossRefzbMATHGoogle Scholar
- Farsiu S, Robinson MD, Elad M, Milanfarv P (2004) Fast and robust multiframe super-resolution. IEEE Trans Image Process 13:1327–1344CrossRefGoogle Scholar
- Gronskis A, Heitz D, Mémin É (2013) Inflow and initial conditions for direct numerical simulation based on adjoint data assimilation. J Comput Phys 242:480–497. doi: 10.1016/j.jcp.2013.01.051 CrossRefGoogle Scholar
- Hardie RC, Barnard KJ, Armstrong EA (1997) Joint map registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans Image Process 6:1621–1633CrossRefGoogle Scholar
- Héas P, Mémin E, Heitz D, Mininni P (2012) Power laws and inverse motion modeling: application to turbulence measurements from satellite images. Dyn Meteorol Oceanogr 64:10962. doi: 10.3402/tellusa.v64i0.10962 Google Scholar
- Heitz D, Mémin E, Schnörr C (2010) Variational fluid flow measurements from image sequences: synopsis and perspectives. Exp Fluids 48(3):369–393. doi: 10.1007/s00348-009-0778-3 CrossRefGoogle Scholar
- Horn B, Schunck B (1981) Determining optical flow. Artif Intell 17(173):185–203CrossRefGoogle Scholar
- Jullien M-C, Castiglione P, Tabeling P (2001) Intermittency of a passive tracer in the inverse energy cascade. Phys Rev E 64(3):035301. doi: 10.1103/PhysRevE.64.035301 CrossRefGoogle Scholar
- Keane R, Adrian R (1992) Theory of cross-correlation analysis of piv images. Appl Sci Res 49(3):191–215CrossRefGoogle Scholar
- Kim K, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133CrossRefMathSciNetGoogle Scholar
- Kurganov A, Tadmor E (2000) New high-resolution central schemes for nonlinear conservation laws and convection? diffusion equations. J Comput Phys 160(1):241–282CrossRefMathSciNetzbMATHGoogle Scholar
- Le Dimet F-X, Talagrand O (1986) Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects. Tellus 38A:97–110CrossRefGoogle Scholar
- Lions J (1971) Optimal control of systems governed by PDEs. Springer, NewYorkGoogle Scholar
- Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. IJCAI 81:674–679Google Scholar
- Moin P, Squires K, Cabot W, Lee S (1991) A dynamic subgrid-scale model for compressible turbulence and scalar transport. Phys Fluids 3(11):2746–2757CrossRefzbMATHGoogle Scholar
- Nasrollahi K, Moeslund T (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468CrossRefGoogle Scholar
- Papadakis N, Corpetti T, Mémin E (2007) Dynamically consistent optical flow estimation. In: IEEE 11th international conference on computer vision (ICCV’07). Rio de Janeiro, Brazil, pp 1–7. doi: 10.1109/ICCV.2007.4408889
- Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36. doi: 10.1109/msp.2003.1203207 CrossRefGoogle Scholar
- Peleg T, Elad M (2014) A statistical prediction model based on sparse representations for single image super-resolution. Image Process IEEE Trans 23(6):2569–2582CrossRefMathSciNetGoogle Scholar
- Purkait P, Chanda B (2012) Super resolution image reconstruction through Bregman iteration using morphologic regularization. Image Process IEEE Trans 21(9):4029–4039CrossRefMathSciNetGoogle Scholar
- Rabaud V, Belongie S (2005) Big little icons. CVPR 3:24–30Google Scholar
- Sagaut P (2006a) Large eddy simulation for incompressible flows. Springer editionGoogle Scholar
- Sagaut P (2006b) Multiscale and multiresolution approaches in turbulence. Imperial College Press, LondonCrossRefzbMATHGoogle Scholar
- Su LK, Dahm WJA (1996a) Scalar imaging velocimetry measurements of the velocity gradient tensor field in turbulent flows. I. Assessment of errors. In: Proceedings of imaging understanding workshopGoogle Scholar
- Su LK, Dahm WJA (1996b) Scalar imaging velocimetry measurements of the velocity gradient tensor field in turbulent flows. II. Experimental results. Phys Fluids 8(7):1883–1906CrossRefGoogle Scholar
- Sun D, Roth S, Darmstadt T (2010) Secrets of optical flow estimation and their principles. In: Proceedings of 2010 IEEE computer vision and pattern recognition, pp 2432–2439Google Scholar
- Sun J, Xu Z, Shum H-Y (2008) Image super-resolution using gradient profile prior. In: CVPR. IEEE Computer Society. ISBN 978-1-4244-2242-5. http://dblp.uni-trier.de/db/conf/cvpr/cvpr2008.html#SunXS08
- Talagrand O, Courtier P (1987) Variational assimilation of meteorological observations with the adjoint vorticity equation I: theory. Q J R Meteorol Soc 113(478):1311–1328CrossRefGoogle Scholar
- Vidard P, Blayo E, Le Dimet F-X, Piacentini A (2000) 4d variational data analysis with imperfect model. Flow Turbul Combust 65(3–4):489–504CrossRefzbMATHGoogle Scholar
- Wen S, Yan L, Feng W, Shipeng L (2009) Real-time screen image scaling and its GPU acceleration. In: ICIP. IEEE, pp 3285–3288. ISBN 978-1-4244-5654-3. http://dblp.uni-trier.de/db/conf/icip/icip2009.html#SunLWL09a
- Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. Image Process IEEE Trans 19(11):2861–2873CrossRefMathSciNetGoogle Scholar
- Zhou F, Yang W, Liao Q (2012) Interpolation-based image super-resolution using multisurface fitting. IEEE Trans Image Process 21(7):3312–3318CrossRefMathSciNetGoogle Scholar
- Zille P, Corpetti T, Shao L, Chen X (2014) Observation model based on scale interactions for optical flow estimation. Image Process IEEE Trans 23(8):3281–3293CrossRefGoogle Scholar