Confounds in the visual stimulus

Separating the Reflectance from the Illuminant

Most of the interesting objects in the world do not themselves emit light, but rather reflect some proportion of whatever light falls on them. To characterize such objects, one needs to determine their reflectance properties, but the amount of light reaching the eye from an object is a product of the illuminant and the reflectance of the object. Since one does not have any independent knowledge of the illuminant, this is one equation with two unknowns and thus formally unsolvable. That is to say, any visual stimulus is massively underspecified, there being an infinite number of possible combinations of surfaces and illuminants that could have produced any given number of photons coming from a particular direction. The severity of this problem is increased by the fact that variations due to the reflectance characteristics of objects are tiny compared to the huge variations in the illuminant. The reflectances of objects vary only over a range of about 20 to 1, with a white object reflecting about 90% of the incident light and a black object about 5%. The variation in light level in the course of a day, on the other hand, can be 1 billion to 1. The daunting task for the visual system is to capture the tiny variations due to objects and separate them from the massive variation due to the illuminant.

A related aspect of this problem is the need to separate the wavelength characteristics of objects from those of the illuminant, for the wavelength distribution of the light coming from an object is also a product of the wavelength distribution of the illuminant and the spectral reflectance characteristics of the object. The light reaching us from the sun is reddish at dawn and dusk and bluer at midday. Thus, both the color and the lightness of objects are indeterminate in the absence of knowledge of the characteristics of the illuminant.

To solve these problems and others, the visual system must in effect make (educated) a priori assumptions about the nature of the world—for instance, about which variations in the stimulus are most likely due to the illuminant (and thus to be largely ignored) and which are likely due to visual objects (and thus to be further processed). Most of these assumptions have been acquired through evolution, others through experience with the environment in the course of development. Many of the characteristics of the neural processes can be seen as reflecting the particular assumptions the visual system makes about the world.

Separating Intensity and Color

A second formally unsolvable problem for the visual nervous system is a consequence of the characteristics of the photopigments that constitute the first stage in the visual neural process. The photopigments have very broad spectral sensitivity; the two major cone types [the L (long-wavelength-sensitive) and M (middle-wavelength-sensitive) cones] absorb light of wavelengths across the whole visible spectrum. A given cone is more likely to absorb light of some wavelengths than of others, but an absorbed photon of any wavelength has precisely the same effect as one of any other wavelength. Thus, at the very first stage, wavelength and intensity information are totally confounded. This is known as the principle of univariance: a receptor is able to signal only one number—the number of photons absorbed; thus, it cannot separately report the wavelength and the intensity of the light incident upon it.