Encyclopedia of Color Science and Technology

2016 Edition
| Editors: Ming Ronnier Luo

Color Scene Statistics, Chromatic Scene Statistics

  • Yoko Mizokami
Reference work entry
DOI: https://doi.org/10.1007/978-1-4419-8071-7_212

Synonyms

Definition

Color scene statistics or chromatic scene statistics are statistical characteristics of scene color. There are a number of ways to analyze color scenes (natural scenes) statistically in relation to vision, such as the average color, color distribution, Bayesian model, principal component analysis, and probabilistic models. Color scene statistics are closely related to the evolution and development of the color vision mechanism at every level of the visual system.

Overview

Color vision has evolved with the natural environment. Thus, the color statistics of a natural scene must have an enormous impact on the evolution and development of the color vision mechanism. There have been many attempts to analyze natural scene statistics and find the connection to the color vision mechanism as well as other aspects of the visual mechanism. Traditionally, the average color and color distribution of a scene are associated with color adaptation and color constancy. Based on those statistical data, physiological, empirical, and statistical models of color constancy were introduced. Other aspects of visual perception, such as color discrimination or color appearance, would be formed by the environment to obtain the best performance or representation under a given color environment. Measurement techniques, such as two- and three-dimensional measurements of scenes and multispectral measurement, provide additional information for deeper analysis or analysis from different perspectives. Powerful computational analyses introduce new statistical approaches for investigating the mechanism of color vision. Here, the topics of color scene statistics related to color vision, visual perception, and their evolution are discussed.

Measurement of Objects, Illumination, and Scenes

Obtaining accurate color information of scenes is important to analyze color scene statistics. Camera calibration techniques and calibrated images in which the RGB values of images are transferable to information on chromaticity coordinates have been developed.

Another important aspect for analyzing colors in natural scenes is the spectral information. RGB information has only three dimensions, and consequently, some information is lost. For instance, RGB information cannot differentiate metameric color, which results when a pair of stimuli exhibit the same apparent color but with different spectral power distributions. Many groups have measured the daylight spectra and spectral reflectance of natural and printing surfaces. Published databases (e.g., SOCS [1]) and personal databases on websites are available.

For obtaining the spectral color information of an entire scene, multispectral or hyperspectral image capture techniques using a multispectral camera are used. Those data are used for studies on color vision mechanisms. It is difficult, however, to obtain a hyperspectral image manually because it takes a long time and the scene must be still (no moving objects or wind) to capture an image with multiple wavelengths by changing bandpass filters. Therefore, collections of multispectral images of outdoor scenes are limited. Recent development of fast and/or accurate multispectral cameras will allow further statistical analysis based on a large number of multispectral data sets.

Although the multispectral information is important and useful, it includes a massive amount of data that is inconvenient to handle. Thus, there have been many attempts to represent multidimensional data with a smaller number of dimensions by compressing the amount of information using principal component analysis (PCA). The natural spectra can be represented well by three basis functions. Those for the illuminant obtained by Judd [2] are shown in Fig. 1. The basis functions give the relative spectral radiant power distribution of the CIE daylight illuminant at different color temperatures. Cohen [3] applied this analysis to the reflectance of Munsell color paper and showed three basis functions that were different from those obtained by Judd. Later studies suggested that Cohen’s data were applicable to natural surfaces (see Fig. 1) and that three basis functions were necessary and probably sufficient for representing the spectral reflectance functions of natural objects. (Note that some studies suggested that a larger number of basis functions are needed depending on the context.) These results led to the hypothesis that the human visual mechanism has a similar representation in the visual process [4]. However, it was recently suggested that Gaussian fitting with three variables to natural spectra (see Fig. 1) is as good as the fitting by three basis functions [5]. More investigations would be needed to determine how human visual system represents the spectral world.
Color Scene Statistics, Chromatic Scene Statistics, Fig. 1

Examples of three basis functions for daylight (left) and reflectance (center), respectively. A Gaussian fitting function with three parameters: peak, amplitude, and bandwidth (for both illumination and reflectance) is also shown (right)

Analysis of Natural Color Statistics

There are a number of ways to analyze the color properties of a scene statistically. The most basic method would be by average color and color distribution. Analysis taking into account spatial frequency is also important because the spatial contrast sensitivities of luminance, and the L-M (reddish–greenish) and S-LM (bluish–yellowish) opponent-color channels are different. Those of chromatic channels tend to have a low-pass shape, suggesting the low-frequency component contributes more to color perception. Recent developments in statistical modeling, along with powerful computational tools, have enabled researchers to study more sophisticated statistical models of visual images and to evaluate these models empirically against large data sets [6].

Based on the different information obtained from color scene statistics, the mechanisms of vision and color perception have been investigated and revealed from aspects of the peripheral to central visual system to perspectives on evolution, development, and short- to long-term adaptation.

Effect on Evolution of Color Vision

The sensitivity of the photoreceptors and how the cone signals are combined in postreceptoral channels have been explained by assuming that they optimize the efficient coding of natural color signals and that they are tuned to specific signals.

For example, it has been suggested that trichromacy in Old World primates and the reflectance functions of tropical fruits coevolved and that primate color vision has been shaped by the need to find reddish (ripe) fruit or young (edible) leaves among green foliage, as shown by the example in Fig. 2. For instance, the tuning of the L and M cones and the later opponent-color system (L-M) based on the signal representing their difference are optimized to detect the color signals provided by ripening fruit or edible foliage [7]. This hypothesis is a good example of how color statistics of the visual environment affected the early-stage evolution of the color vision system.
Color Scene Statistics, Chromatic Scene Statistics, Fig. 2

Examples of the advantage of trichromacy (top), capable of redgreen discrimination, compared to dichromacy (bottom) on finding red fruits among green leaves

It has also been shown that seasonal variations in the color statistics of natural images alter both the average color and color distribution in scenes, as shown in Fig. 3. On opponent-color space, arid periods are marked by a mean shift toward the + L pole of the L-M chromatic axis. A rotation in the color distributions away from the S-LM chromatic axis and toward an axis of bluish–yellowish variation, both primarily due to changes in vegetation, implies that these changes contribute to the construction of the mechanism of both visual sensitivity and color appearance [8].
Color Scene Statistics, Chromatic Scene Statistics, Fig. 3

An example of seasonal changes in the natural color scene. (Mountain forest in Sierra Nevada)

There are other hypotheses, such as skin tone discrimination and predator detection affecting evolution. There are a number of variations or changes in natural environments other than the examples shown above, which may contribute to formation of the color vision mechanism. It is interesting to pursue how the color vision mechanism was optimized during evolution and development [9].

Cortical Mechanism and Probabilistic Approaches

Cortical color-coding is also thought to have evolved to represent important characteristics of the structure of color in the environment. Probabilistic modeling allows us to test experimentally the efficient coding hypothesis for both individual neurons and populations of neurons [6].

“Sparse coding,” the concept that neurons encode sensory information using a small number of active neurons at any given second, has been introduced. It increases the storage capacity in associative memories and saves energy. It also makes the structure in natural signals explicit and makes complex data easier to interpret during visual processing [10]. Figure 4 shows an example of a set of basis functions derived from a set of learning images including natural and man-made scenes based on a sparse coding model. Any natural image (scene) can be constructed using combinations of basis functions. This suggests that the human visual system also constructs the image of a natural scene based on the activity of neurons having receptive fields with different properties.
Color Scene Statistics, Chromatic Scene Statistics, Fig. 4

An example of sparse coding created by a procedure in accordance with the method in the paper of Olshausen and Field [10]. Right, image-set for learning; Left, derived basis functions

Probabilistic models are also applied to the chromatic structure of natural scenes. Independent component analysis (ICA) has been applied to natural color images to establish an efficient representation of color in natural scenes. It was shown to produce achromatic and color-opponent basis functions as well as spatiochromatic independent components of cortical neurons. This suggests a relationship between statistics of natural scenes and cortical color processing.

Effect of Color Statistics of a Scene on Color Constancy

Color constancy is a phenomenon in which the stability of the color appearance of an object surface is maintained among variations in the illuminating environment. The central issue is always how humans separate illumination and the surface of objects, which is a necessary task for achieving color constancy. Identical to other mechanisms in earlier stages of the visual system, color constancy is also associated with the color statistics of environments [4, 9, 11].

A classic approach is the von Kries adaptation. To set a neutral point (or illumination color) for application of the von Kries adaptation, many assumptions or hypotheses based on scene statistics have been proposed (e.g., average color of a scene: the gray world assumption). In Bayesian decision theory, which is a more statistical approach, the most likely combination of illuminants and surfaces is determined based on prior distributions that describe the probability that particular illuminants and surfaces exist in the world.

Characteristics of a scene statistics also contribute to color constancy [11, 12]. One question is how humans can tell the difference between a white paper under red illumination and a red paper under white illumination if the papers have the same chromaticity. It was suggested that the luminance of the white paper decreases statistically under the red light and that this relationship could be a clue for differentiating the two. Besides, in natural scenes, chromatic variations and the luminance variations aligned with them mainly arise from object surfaces such as the border between different materials, whereas pure or near-pure luminance variations mainly arise from inhomogeneous illumination such as shadows or shading. The human visual system uses these color–luminance relationships and determines the three-dimensional structure of a scene from the natural relationships that exist between color and luminance in the visual scene, material surface, and so on. It is suggested that natural scene statistics could also be the universal basis of color context effects, such as color contrast and color assimilation.

Adaptation to the Color of Natural Scenes

As shown in color constancy, it is well known that color perception adapts to the hue change of the color distribution. Color appearance is also influenced by the saturation or variance of the color distribution of a scene. A pale color patch appears less colorful when it is surrounded by saturated colors, and chromatically selective compression along any direction of chromatic variation occurs after adapting to temporal color variation. These suggest adaptation to the specific color gamut within individual scenes and natural environments [9, 11]. It is also shown that the impression of a natural image shifts to being less colorful after adaptation to a series of saturated images and vice versa [13]. Moreover, it is suggested that simple statistics, such as the color distribution itself, cannot explain the effect because the adaptation is stronger for natural images than scrambled images. It implies the contribution of higher-level mechanisms, such as scene recognition or cognition.

The timescale of adaptation varies. It can be seconds, minutes, hours, months, years, and possibly a lifetime. The tuning of sensitivities of the color vision mechanism likely continues in all timescales. Evolution and development of the visual function could be considered very long-term adaptation. It is also suggested that more cognitive or social aspects of color perception, such as color category, could be influenced by the color scene statistics of each region. One of the functional meanings of adaptation would be compensating for variations within an observer and between observers as well as within an environment and between environments, in order to maintain a stable and coherent color appearance. The color scene statistics certainly contribute to it. The adaptation shaped by environmental pressure would achieve an efficient transmission and stability of information from the periphery in the visual system to the centers in the brain.

Cross-References

References

  1. 1.
    Standard object colour spectra database for colour reproduction evaluation (SOCS). TR X 0012:2004 (ISO/TR 16066:2003), Japanese Standards Association (2004)Google Scholar
  2. 2.
    Wyszecki, G., Stiles, W.S.: Color Science. Wiley, New York (1982)Google Scholar
  3. 3.
    Cohen, J.: Dependency of the spectral reflectance curves of the Munsell color chips. Psychon. Sci. 1, 369–370 (1964)CrossRefGoogle Scholar
  4. 4.
    Foster, D.H.: Color constancy. Vision Res. 51, 674–700 (2011)CrossRefGoogle Scholar
  5. 5.
    Mizokami, Y., Webster, M.A.: Are Gaussian spectra a viable perceptual assumption in color appearance? J. Opt. Soc. Am. A Opt. Image Sci. Vis. 29, A10–A18 (2012)ADSCrossRefGoogle Scholar
  6. 6.
    Simoncelli, E.P., Olshausen, B.A.: Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001)CrossRefGoogle Scholar
  7. 7.
    Osorio, D., Vorobyev, M.: A review of the evolution of animal colour vision and visual communication signals. Vision Res. 48, 2042–2051 (2008)CrossRefGoogle Scholar
  8. 8.
    Webster, M.A., Mizokami, Y., Webster, S.M.: Seasonal variations in the color statistics of natural images. Network 18, 213–233 (2007)CrossRefGoogle Scholar
  9. 9.
    Webster, M.A.: Adaptation and visual coding. J. Vis. 11(5), 3, 1–23 (2011)Google Scholar
  10. 10.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)ADSCrossRefGoogle Scholar
  11. 11.
    MacLeod, D.I.A., Golz, J.: A computational analysis of colour constancy. In: Mausfeld, R., Heyer, D. (eds.) Colour Perception: Mind and the Physical World, pp. 205–246. Oxford University Press, London (2003)CrossRefGoogle Scholar
  12. 12.
    Shevell, S.K., Kingdom, F.A.A.: Color in complex scenes. Annu. Rev. Psychol. 59, 143–166 (2008)CrossRefGoogle Scholar
  13. 13.
    Mizokami, Y., Kamesaki, C., Ito, N., Sakaibara, S., Yaguchi, H.: Effect of spatial structure on colorfulness adaptation for natural images. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 29, A118–A127 (2012)ADSCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Graduate School of Advanced Integration ScienceChiba UniversityChibaJapan