International Journal of Computer Vision

, Volume 54, Issue 1–3, pp 83–103 | Cite as

The Nonlinear Statistics of High-Contrast Patches in Natural Images

  • Ann B. Lee
  • Kim S. Pedersen
  • David Mumford


Recently, there has been a great deal of interest in modeling the non-Gaussian structures of natural images. However, despite the many advances in the direction of sparse coding and multi-resolution analysis, the full probability distribution of pixel values in a neighborhood has not yet been described. In this study, we explore the space of data points representing the values of 3 × 3 high-contrast patches from optical and 3D range images. We find that the distribution of data is extremely “sparse” with the majority of the data points concentrated in clusters and non-linear low-dimensional manifolds. Furthermore, a detailed study of probability densities allows us to systematically distinguish between images of different modalities (optical versus range), which otherwise display similar marginal distributions. Our work indicates the importance of studying the full probability distribution of natural images, not just marginals, and the need to understand the intrinsic dimensionality and nature of the data. We believe that object-like structures in the world and the sensor properties of the probing device generate observations that are concentrated along predictable shapes in state space. Our study of natural image statistics accounts for local geometries (such as edges) in natural scenes, but does not impose such strong assumptions on the data as independent components or sparse coding by linear change of bases.

natural image statistics non-linear sparse coding pixel-based image models microimages clutter higher-order statistics geometrically based statistics high-dimensional probability density estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bronshtein, I.N. and Semendyayev, K.A. 1998. Handbook of Mathematics. Springer-Verlag, 3rd ed.Google Scholar
  2. Buccigrossi, R.W. and Simoncelli, E.P. 1999. Image compression via joint statistical characterization in the wavelet domain. IEEE Trans Image Processing, 8(12):1688-1701.Google Scholar
  3. Conway, J.H. and Sloane, N.J.A. 1988. Sphere Packings, Lattices and Groups, No. 290 in Grundlehren der mathematischen Wissenschaften. Springer-Verlag.Google Scholar
  4. Cover, T.M. and Thomas, J.A. 1991. Elements of Information Theory. New York: John Wiley & Sons.Google Scholar
  5. Elliott, J.P. and Dawber, P.G. 1979. Symmetry in Physics, vol. 1, New York: Oxford University Press.Google Scholar
  6. Field, D.J. 1987. Relations between the statistics of natural images and the response properties of cortical cells. Journal of Optical Society of America 4(12):2379-2394.Google Scholar
  7. Friedman, J.H. 1987. Exploratory projection pursuit. Journal of the American Statistical Association, 82(397):249-266.Google Scholar
  8. Gemen, D. and Koloydenko, A. 1999. Invariant statistics and coding of natural microimages. In Proc. of the IEEE Workshop on Statistical and Computational Theories of Vission. Published on the Web.Google Scholar
  9. Grenander, U. and Srivastava, A. 2001. Probability models for clutter in natural images. IEEE Trans. PAMI 23(4):424-429.Google Scholar
  10. Huang, J., Lee, A.B., and Mumford, D. 2000. Statistics of range images. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1. Hilton Head Island, SC, pp. 324-331.Google Scholar
  11. Huang, J. and Mumford, D. 1999. Statistics of natural images and models. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition. Google Scholar
  12. Huber, P.J. 1985. Projection pursuit. The Annals of Statistics, 13(2):435-475.Google Scholar
  13. Hyvärinen, A. 1999. Survey on independent component analysis. Neural Computing Surveys, 2:94-128.Google Scholar
  14. Malik, J., Belongie, S., Leung, T., and Shi, J. 2001. Contour and texture analysis for image segmentation. International Journal of Computer Vision, 43(1):7-27.Google Scholar
  15. Marr, D. 1982. Vision. New York: W.H. Freeman.Google Scholar
  16. Nielsen, M. and Lillholm, M. 2001. What do features tell about images? In Scale-Space and Morphology in Computer Vision, M. Kerckhove (Ed.). pp. 39-50.Google Scholar
  17. Olshausen, B.A. and Field, D.J. 1996. Natural image statistics and efficient coding. Network: Computation in Neural Systems, 7(2):333-339.Google Scholar
  18. Reinagel, P. and Zador, A.M. 1999. Natural scene statistics at the centre of gaze. Network: Computation in Neural Systems, 10(4):341-350.Google Scholar
  19. Ruderman, D.L. and Bialek, W. 1994. Statistics of natural images: Scaling in the woods. Physical Review Letters, 73(6):814- 817.Google Scholar
  20. Simoncelli, E.P. 1999a. Bayesian denoising of visual images in the wavelet domain. In Bayesian Inference in Wavelet Based Models, P. Müller and B. Vidakovic (Eds.). New York: Springer-Verlag, pp. 291-308.Google Scholar
  21. Simoncelli, E.P. 1999b. Modeling the joint statistics of images in the wavelet domain. In Proc. SPIE, 44th Annual Meeting, vol. 3813. Denver, CO. pp. 188-195.Google Scholar
  22. Sullivan, J., Blake, A., Isard, M., and MacCormick, J. 1999. Object localization by bayesian correlation. In Proc. Int. Conf. Computer Vision. Corfu, Greece, pp. 1068-1075.Google Scholar
  23. Tu, Z.W., Zhu, S.C., and Shum, H.Y. 2001. Image segmentation by data driven markov chain Monte Carlo. In Proc. of International Conference on Computer Vision. Vancouver, Canada.Google Scholar
  24. van Hateren, J.H. and van der Schaaf, A. 1998. Independent component filters of natural images compared with simple cells in primary visual cortex. In Proc. R. Soc. Lond., vol. B 265, pp. 359- 366.Google Scholar
  25. Wegmann, B. and Zetzsche, C. 1990. Statistical dependence between orientation filter outputs used in an human vision based image code. In Proc. SPIE Visual Comm. and Image Processing, vol. 1360. Lausanne, Switzerland, pp. 909-922.Google Scholar
  26. Zhu, S.C. and Mumford, D. 1998. GRADE: Gibbs reaction and diffusion equations-a framework for pattern synthesis, image denoising, and removing clutter. In Proc. of International Conference on Computer Vision.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Ann B. Lee
    • 1
  • Kim S. Pedersen
    • 2
  • David Mumford
    • 3
  1. 1.Division of Applied MathematicsBrown UniversityProvidenceUSA
  2. 2.Department of Computer ScienceUniversity of CopenhagenCopenhagen ØDenmark
  3. 3.Division of Applied MathematicsBrown UniversityProvidenceUSA

Personalised recommendations