Journal of Mathematical Imaging and Vision

, Volume 60, Issue 8, pp 1231–1245 | Cite as

Highly Corrupted Image Inpainting Through Hypoelliptic Diffusion

  • Ugo V. Boscain
  • Roman ChertovskihEmail author
  • Jean-Paul Gauthier
  • Dario Prandi
  • Alexey Remizov


We present a new biomimetic image inpainting algorithm, the Averaging and Hypoelliptic Evolution (AHE) algorithm, inspired by the one presented in Boscain et al. (SIAM J. Imaging Sci. 7(2):669–695, 2014) and based upon a semi-discrete variation of the Citti–Petitot–Sarti model of the primary visual cortex V1. The AHE algorithm is based on a suitable combination of sub-Riemannian hypoelliptic diffusion and ad hoc local averaging techniques. In particular, we focus on highly corrupted images (i.e., where more than the 80% of the image is missing), for which we obtain high-quality reconstructions.


Image reconstruction Inpainting Sub-Riemannian hypoelliptic diffusion 



We deeply thank G. Facciolo, S. Masnou and G.P. Panasenko for their help. This work was supported by the ERC POC project ARTIV1 contract number 727283, by the ANR project “SRGI” ANR-15-CE40-0018, by a public grant as part of the Investissement d’avenir project, reference ANR-11-LABX-0056-LMH, LabEx LMH (in a joint call with Programme Gaspard Monge en Optimisation et Recherche Opérationnelle), by the iCODE institute, research project of the Idex Paris-Saclay, by the POCI-01-0145-FEDER-006933/SYSTEC project financed by ERDF and FCT through COMPETE2020.


  1. 1.
    Abas, F.: Analysis of craquelure patterns for content-based retrieval. PhD thesis, University of Southampton (2004)Google Scholar
  2. 2.
    Agrachev, A., Barilari, D., Boscain, U.: Introduction to Riemannian and Sub-Riemannian Geometry (Lecture Notes).
  3. 3.
    Bellaïche, A.: The tangent space in sub-Riemannian geometry. In: Sub-Riemannian Geometry, volume 144 of Progress in Mathematics, pp. 1–78. Birkhäuser, Basel (1996)Google Scholar
  4. 4.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH 2000, New Orleans, USA, pp. 417–424 (2000)Google Scholar
  5. 5.
    Bohi, A., Prandi, D., Guis, V., Bouchara, F., Gauthier, J.-P.: Fourier descriptors based on the structure of the human primary visual cortex with applications to object recognition. J. Math. Imaging Vis. 57(1), 117–133 (2017)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Boscain, U., Charlot, G., Rossi, F.: Existence of planar curves minimizing length and curvature. Proc. Steklov Inst. Math. 270, 43–56 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Boscain, U., Chertovskih, R.A., Gauthier, J.-P., Remizov, A.O.: Hypoelliptic diffusion and human vision: a semidiscrete new twist. SIAM J. Imaging Sci. 7(2), 669–695 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Boscain, U., Duits, R., Rossi, F., Sachkov, Yu.: Curve cuspless reconstruction via sub-Riemannian geometry. ESAIM Control Optim. Calc. Var. 20(3), 748–770 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Boscain, U., Duplaix, J., Gauthier, J.-P., Rossi, F.: Anthropomorphic image reconstruction via hypoelliptic diffusion. SIAM J. Control Optim. 50(3), 1–25 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Boscain, U., Gauthier, J.-P., Prandi, D., Remizov, A.: Image reconstruction via non-isotropic diffusion in Dubins/Reed-Shepp-like control systems. In: 53rd IEEE Conference on Decision and Control, pp. 4278–4283 (2014)Google Scholar
  11. 11.
    Bugeau, A., Bertalmío, M., Caselles, V., Sapiro, G.: A comprehensive framework for image inpainting. IEEE Trans. Image Process. 19(10), 2634–2645 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Chan, T.F., Kang, S.H., Shen, J.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Citti, G., Franceschiello, B., Sanguinetti, G., Sarti, A.: Sub-Riemannian mean curvature flow for image processing. SIAM J. Imaging Sci. 9(1), 212–237 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Citti, G., Sarti, A.: A cortical based model of perceptual completion in the roto-translation space. J. Math. Imaging Vis. 24(3), 307–326 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Cornelis, B., Ružić, T., Gezels, E., Dooms, A., Pižurica, A., Platiša, L., Cornelis, J., Martens, M., De Mey, M., Daubechies, I.: Crack detection and inpainting for virtual restoration of paintings: the case of the ghent altarpiece. Signal Process. 93(3), 605–619 (2013)CrossRefGoogle Scholar
  16. 16.
    Duits, R., Boscain, U., Rossi, F., Sachkov, Y.: Association fields via cuspless sub-Riemannian geodesics in SE(2). J. Math. Imaging Vis. 49(2), 384–417 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Duits, R., Franken, E.: Left-invariant parabolic evolutions on \({\rm SE}(2)\) and contour enhancement via invertible orientation scores. Part I: linear left-invariant diffusion equations on \({\rm SE}(2)\). Q. Appl. Math. 68(2), 255–292 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Duits, R., Franken, E.: Left-invariant parabolic evolutions on \({\rm SE}(2)\) and contour enhancement via invertible orientation scores. Part II: nonlinear left-invariant diffusions on invertible orientation scores. Q. Appl. Math. 68(2), 293–331 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Duits, R., van Almsick, M.A.: The explicit solutions of linear left-invariant second order stochastic evolution equations on the 2D euclidean motion group. Q. Appl. Math. 66, 27–67 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Facciolo, G., Arias, P., Caselles, V., Sapiro, G.: Exemplar-based interpolation of sparsely sampled images. In: Cremers, D., Boykov, Yu., Blake, A., Schmidt, F.R. (eds.) Energy Minimization Methods in Computer Vision and Pattern Recognition: 7th International Conference EMMCVPR 2009 (Bonn, Germany, August 24–27, 2009) Proceedings, pp. 331–344. Springer (2009)Google Scholar
  21. 21.
    Ferziger, J.H., Perić, M.: Computational Methods for Fluid Dynamics, 3rd rev. edn. Springer, Berlin (2002)Google Scholar
  22. 22.
    Gromov, M.: Carnot-Carathéodory spaces seen from within. In: Sub-Riemannian Geometry, volume 144 of Progress in Mathematics, pp. 79–323. Birkhäuser, Basel (1996)Google Scholar
  23. 23.
    Hladky, R.K., Pauls, S.D.: Minimal surfaces in the roto-translation group with applications to a neuro-biological image completion model. J. Math. Imaging Vis. 36(1), 1–27 (2010)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Hörmander, L.: Hypoelliptic second order differential equations. Acta Math. 119, 147–171 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Hubel, D.H., Wiesel, T.N.: Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148, 574–591 (1959)CrossRefGoogle Scholar
  26. 26.
    Marchuk, G.I.: Methods of Numerical Mathematics. Springer, Berlin (1982)CrossRefzbMATHGoogle Scholar
  27. 27.
    Marr, D., Hildreth, E.: Theory of edge detection. Proc. R. Soc. Lond. B Biol. Sci. 207(1167), 187–217 (1980)CrossRefGoogle Scholar
  28. 28.
    Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. Image Process. 11(2), 68–76 (2002)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Masnou, S., Morel, J.-M.: Level lines based disocclusion. In: Proceedings of 5th IEEE International Confernce on Image Processing, vol. 3, pp. 259–263 (1998)Google Scholar
  30. 30.
    Montgomery, R.: A Tour of Subriemannian Geometries, Their Geodesics and applications, volume 91 of Mathematical Surveys and Monographs. American Mathematical Society, Providence, RI (2002)Google Scholar
  31. 31.
    Peichl, L., Wässle, H.: Size, scatter and coverage of ganglion cell receptive field centres in the cat retina. J. Physiol. 291, 117–141 (1979)CrossRefGoogle Scholar
  32. 32.
    Petitot, J.: The neurogeometry of pinwheels as a sub-Riemannian contact structure. J. Physiol. Paris 97(2–3), 265–309 (2003)CrossRefGoogle Scholar
  33. 33.
    Petitot, J.: Neurogéométrie de la vision - Modèles mathématiques et physiques des architectures fonctionnelles. Les Éditions de l’École Polytechnique (2008)Google Scholar
  34. 34.
    Ponomarenko, N., Jin, L., Lukin, V., Egiazarian, K.: Self-similarity measure for assessment of image visual quality. In: Proceedings of the 13th International Conference on Advanced Concepts for Intelligent Vision Systems, ACIVS’11, pp. 459–470, Berlin, Heidelberg (2011)Google Scholar
  35. 35.
    Prandi, D., Boscain, U., Gauthier, J.-P.: Image processing in the semidiscrete group of rototranslations. In: Geometric Science of Information, volume 9389 of Lecture Notes in Computer Science, pp. 627–634. Springer (2015)Google Scholar
  36. 36.
    Prandi, D., Gauthier, J.-P.: A semidiscrete version of the Petitot model as a plausible model for anthropomorphic image reconstruction and pattern recognition. Springer Briefs in Mathematics. Springer. arXiv:1704.03069
  37. 37.
    Sanguinetti, G., Citti, G., Sarti, A.: Image completion using a diffusion driven mean curvature flow in a sub-Riemannian space. In: Proceedings of the 3rd International Conference on Computer Vision Theory and Applications (VISAPP 2008), volume 2, pp. 46–53 (2008)Google Scholar
  38. 38.
    Strichartz, R.S.: Sub-Riemannian geometry. J. Differ. Geom. 24(2), 221–263 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Strichartz, R.S.: Corrections to: Sub-Riemannian geometry [J. Differential Geom. 24(2), 221–263 (1986); MR0862049 (88b:53055)]. J. Differ. Geom. 30(2), 595–596 (1989)Google Scholar
  40. 40.
    Voronin, V.V., Frantc, V.A., Marchuk, V.I., Sherstobitov, A.I., Egiazarian, K.: No-reference visual quality assessment for image inpainting. In: Proceedings of SPIE 9399, Image Processing: Algorithms and Systems XIII (March 16, 2015), p. 93990U (2015)Google Scholar
  41. 41.
    Wang, M., Yan, B., Ngan, K.N.: An efficient framework for image/video inpainting. Signal Process. Image Commun. 28(7), 753–762 (2013)CrossRefGoogle Scholar
  42. 42.
    Zhang, F., Li, S., Ma, L., Ngan, K.N.: Limitation and challenges of image quality measurement. In: Proceedings of SPIE 7744, Visual Communications and Image Processing 2010 (July 13, 2010), pp. 774402–774402–8 (2010)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.CNRS, Laboratoire Jacques-Louis LionsUPMC Univ Paris 06ParisFrance
  2. 2.INRIA Team CAGEINRIAParisFrance
  3. 3.Research Center for Systems and Technologies, Faculty of EngineeringUniversity of PortoPortoPortugal
  4. 4.Samara National Research UniversitySamaraRussia
  5. 5.LSIS, UMR CNRS 7296Université de Toulon USTVLa Garde CedexFrance
  6. 6.CNRS, L2SCentraleSupélecGif-sur-YvetteFrance
  7. 7.CNRSCMAP École PolytechniquePalaiseau CedexFrance

Personalised recommendations