Advertisement

Sparse Modeling of Textures

  • Gabriel Peyré
Article

Abstract

This paper presents a generative model for textures that uses a local sparse description of the image content. This model enforces the sparsity of the expansion of local texture patches on adapted atomic elements. The analysis of a given texture within this framework performs the sparse coding of all the patches of the texture into the dictionary of atoms. Conversely, the synthesis of a new texture is performed by solving an optimization problem that seeks for a texture whose patches are sparse in the dictionary. This paper explores several strategies to choose this dictionary. A set of hand crafted dictionaries composed of edges, oscillations, lines or crossings elements allows to synthesize synthetic images with geometric features. Another option is to define the dictionary as the set of all the patches of an input exemplar. This leads to computer graphics methods for synthesis and shares some similarities with non-local means filtering. The last method we explore learns the dictionary by an optimization process that maximizes the sparsity of a set of exemplar patches. Applications of all these methods to texture synthesis, inpainting and classification shows the efficiency of the proposed texture model.

Keywords

Image processing Texture synthesis Sparse representation Learning dictionaries Inpainting 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aharon, M., Elad, M.: Sparse and redundant modeling of image content using an image-signature-dictionary. J. Imag. Sci. 1(3), 228–247 (2008) CrossRefGoogle Scholar
  2. 2.
    Aharon, M., Elad, M., Bruckstein, A.M.: The k-svd: An algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006) CrossRefGoogle Scholar
  3. 3.
    Ashikhmin, M.: Synthesizing natural textures. In: SI3D’01: Proceedings of the 2001 Symposium on Interactive 3D Graphics, pp. 217–226. Assoc. Comput. Mach., New York (2001) CrossRefGoogle Scholar
  4. 4.
    Ballester, C., Caselles, V., Verdera, J.: Disocclusion by joint interpolation of vector fields and gray levels. Multiscale Model. Simul. 2(1), 80–123 (2003) CrossRefMathSciNetGoogle Scholar
  5. 5.
    Bar-Joseph, Z., El-Yaniv, R., Lischinski, D., Werman, M.: Texture mixing and texture movie synthesis using statistical learning. IEEE Trans. Vis. Comput. Graph. 7(2), 120–135 (2001) CrossRefGoogle Scholar
  6. 6.
    Bell, A.J., Sejnowski, T.J.: The independent components of natural scenes are edge filters. Vis. Res. 37, 3327–3338 (1997) CrossRefGoogle Scholar
  7. 7.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proc. of Siggraph 2000, pp. 417–424 (2000) Google Scholar
  8. 8.
    Bovik, A.C.: Analysis of multichannel narrow-band filters for image texture segmentation. IEEE Trans. Signal Process. 39(9), 2025 (1991) CrossRefGoogle Scholar
  9. 9.
    Brox, T., Cremers, D.: Iterated nonlocal means for texture restoration. In: Proc. International Conference on Scale Space and Variational Methods in Computer Vision, Ischia, Italy. LNCS. Springer, Berlin (2007) Google Scholar
  10. 10.
    Buades, A., Coll, B., Morel, J.M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998) CrossRefMathSciNetGoogle Scholar
  12. 12.
    Clausi, D.A., Jernigan, M.E.: Designing Gabor filters for optimal texture separability. Pattern Recogn. 33(11), 1835–1849 (2000) CrossRefGoogle Scholar
  13. 13.
    Cohen, L.D.: A new approach of vector quantization for image data compression and texture detection. In: International Conference on Pattern Recognition, pp. 1250–1254 (1988) Google Scholar
  14. 14.
    Cook, R.L., DeRose, T.: Wavelet noise. ACM Trans. Graph. 24(3), 803–811 (2005) CrossRefGoogle Scholar
  15. 15.
    Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004) CrossRefGoogle Scholar
  16. 16.
    Daubechies, I., Defrise, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57, 1413–1541 (2004) zbMATHCrossRefGoogle Scholar
  17. 17.
    De Bonet, J.S.: Multiresolution sampling procedure for analysis and synthesis of texture images. In: Proc. of SIGGRAPH’97, pp. 361–368. Assoc. Comput. Mach./Addison Wesley, New York/Reading (1997) Google Scholar
  18. 18.
    Donoho, D.: Wedgelets: Nearly-minimax estimation of edges. Ann. Stat. 27, 353–382 (1999) CrossRefMathSciNetGoogle Scholar
  19. 19.
    Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: ICCV’99: Proceedings of the International Conference on Computer Vision, vol. 2, p. 1033. IEEE Computer Society, Los Alamitos (1999) CrossRefGoogle Scholar
  20. 20.
    Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Proc. of SIGGRAPH 2001, pp. 341–346 (2001) Google Scholar
  21. 21.
    Engan, K., Aase, S.O., Hakon Husoy, J.: Method of optimal directions for frame design. In: Proc. ICASSP’99, Washington, DC, pp. 2443–2446. IEEE Computer Society, Los Alamitos (1999) Google Scholar
  22. 22.
    Fadili, M.J., Starck, J.-L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. (2006, revised) Google Scholar
  23. 23.
    Heeger, D.J., Bergen, J.R.: Pyramid-Based texture analysis/synthesis. In: Cook, R. (ed.) SIGGRAPH 95 Conference Proceedings. Annual Conference Series, pp. 229–238. Assoc. Comput. Mach./Addison Wesley, New York/Reading (1995) CrossRefGoogle Scholar
  24. 24.
    Jain, A.K., Farrokhnia, F.: Unsupervised texture segmentation using Gabor filters. Pattern Recogn. 24(12), 1167–1186 (1991) CrossRefGoogle Scholar
  25. 25.
    Julesz, B.: Visual pattern discrimination. IRE Trans. Inf. Theory 8(2), 84–92 (1962) CrossRefGoogle Scholar
  26. 26.
    Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.-W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Comput. 15(2), 349–396 (2003) zbMATHCrossRefGoogle Scholar
  27. 27.
    Kwatra, V., Essa, I., Bobick, A., Kwatra, N.: Texture optimization for example-based synthesis. ACM Trans. Graph. 24(3), 795–802 (2005). Proc. of SIGGRAPH 2005 CrossRefGoogle Scholar
  28. 28.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003). SIGGRAPH 2003 CrossRefGoogle Scholar
  29. 29.
    Lee, D.D., Seung, H.S.: Algorithms for non-negative matrix factorization. In: Advances in Neural Information Processing Systems 13. MIT Press, Cambridge (2001) Google Scholar
  30. 30.
    Lefebvre, S., Hoppe, H.: Parallel controllable texture synthesis. ACM Trans. Graph. 24(3), 777–786 (2005) CrossRefGoogle Scholar
  31. 31.
    Lefebvre, S., Hoppe, H.: Appearance-space texture synthesis. ACM Trans. Graph. 25(3), 541–548 (2006) CrossRefGoogle Scholar
  32. 32.
    Lewicki, M.S., Sejnowski, T.J.: Learning overcomplete representations. Neural Comput. 12(2), 337–365 (2000) CrossRefGoogle Scholar
  33. 33.
    Lu, S.Y., Hernandez, J.E., Clark, G.A.: Texture segmentation by clustering of Gabor feature vectors. In: Proc. IJCNN’91, International Joint Conference on Neural Networks, vol. I, pp. 683–688. IEEE Press, New York (1991) Google Scholar
  34. 34.
    Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008) CrossRefMathSciNetGoogle Scholar
  35. 35.
    Mairal, J., Sapiro, G., Elad, M.: Learning multiscale sparse representations for image and video restoration. Preprint (2007) Google Scholar
  36. 36.
    Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1998) zbMATHGoogle Scholar
  37. 37.
    Manduchi, R., Portilla, J.: Independent component analysis of textures. In: ICCV, pp. 1054–1060 (1999) Google Scholar
  38. 38.
    Matusik, W., Zwicker, M., Durand, F.: Texture design using a simplicial complex of morphable textures. ACM Trans. Graph. 24(3), 787–794 (2005) CrossRefGoogle Scholar
  39. 39.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature 381(6583), 607–609 (1996) CrossRefGoogle Scholar
  40. 40.
    Paragios, N., Deriche, R.: Geodesic active regions and level set methods for supervised texture segmentation. Int. J. Comput. Vis. 46(3), 223–247 (2002) zbMATHCrossRefGoogle Scholar
  41. 41.
    Perlin, K.: An image synthesizer. In: Proc. of SIGGRAPH’85, pp. 287–296. Assoc. Comput. Mach., New York (1985) Google Scholar
  42. 42.
    Peyré, G.: Non-negative sparse modeling of textures. In: Proc. of SSVM’07, pp. 628–639 (2007) Google Scholar
  43. 43.
    Peyré, G.: Manifold models for signals and images. Comput. Vis. Image Underst. (2008, to appear) Google Scholar
  44. 44.
    Portilla, J., Simoncelli, E.P.: A parametric texture model based on joint statistics of complex wavelet coefficients. Int. J. Comput. Vis. 40(1), 49–70 (2000) zbMATHCrossRefGoogle Scholar
  45. 45.
    Sallee, P., Olshausen, B.A.: Learning sparse multiscale image representations. In: Becker, S., Thrun, S., Obermayer, K. (eds.) NIPS, pp. 1327–1334. MIT Press, Cambridge (2002) Google Scholar
  46. 46.
    Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Ann. Rev. Neurosci. 24, 1193–1215 (2001) CrossRefGoogle Scholar
  47. 47.
    Skretting, K., Husoy, J.H.: Texture classification using sparse frame based representations. EURASIP J. Appl. Signal Process. 2006(1), 102–102 (2006) Google Scholar
  48. 48.
    Starck, J.-L., Elad, M., Donoho, D.L.: Redundant multiscale transforms and their application for morphological component analysis. Adv. Imaging Electron Phys. 132, 287–348 (2004) Google Scholar
  49. 49.
    Tropp, J.A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004) CrossRefMathSciNetGoogle Scholar
  50. 50.
    Tropp, J.A.: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 52(3), 1030–1051 (2006) CrossRefMathSciNetGoogle Scholar
  51. 51.
    Tschumperlé, D., Deriche, R.: Vector-valued image regularization with PDEs: A common framework for different applications. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 506–517 (2005) CrossRefGoogle Scholar
  52. 52.
    Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475–494 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  53. 53.
    Tuceryan, M.: Moment-based texture segmentation. Pattern Recogn. Lett. 15(7), 659–668 (1994) CrossRefGoogle Scholar
  54. 54.
    Wei, L.-Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: SIGGRAPH’00: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 479–488. Assoc. Comput. Mach./Addison-Wesley, New York/Reading (2000) CrossRefGoogle Scholar
  55. 55.
    Wei, L.-Y., Han, J., Zhou, K., Bao, H., Guo, B., Shum, H.-Y.: Inverse texture synthesis. ACM Trans. Graph. 27(3), 1–9 (2008). http://doi.acm.org/10.1145/1360612.1360651 Google Scholar
  56. 56.
    Zeng, X.-Y., Chen, Y.-W., van Alphen, D., Nakao, Z.: Selection of ICA features for texture classification. In: ISNN (2), pp. 262–267 (2005) Google Scholar
  57. 57.
    Zhu, S.C., Liu, X.W., Wu, Y.N.: Exploring texture ensembles by efficient Markov chain Monte Carlo-toward a ‘trichromacy’ theory of texture. IEEE Trans. Pattern Anal. Mach. Intell. 22(6), 554–569 (2000) CrossRefGoogle Scholar
  58. 58.
    Zhu, S.C., Wu, Y., Mumford, D.: Filters, random fields and maximum entropy (FRAME): Towards a unified theory for texture modeling. Int. J. Comput. Vis. 27(2), 107–126 (1998) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.CNRS and CeremadeUniversité Paris-DauphineParis Cedex 16France

Personalised recommendations