Visual boundaries and surfaces and complementary cortical streams

Visual perception is highly context-sensitive. A visual scene cannot be easily understood by evaluating it just in terms of its local contrasts or individual pixels. It is better understood as a juxtaposition of objects which may mutually occlude one another in a scene or picture. Much evidence suggests that these objects are represented in terms of two types of information: boundaries and surfaces. This chapter reviews aspects of how and why this happens.

Where are visual boundaries and surfaces computed in the brain? Figure 110.1 summarizes three processing streams within the visual cortex that are activated by light impinging on the retina. One stream goes from the retina through the lateral geniculate nucleus (LGN) parvo stage (classified due to its parvocellular cell type) to the cortical processing stages V1 interblob, V2 interstripe, V4, and then to inferotemporal cortex. Another stream goes from retina through LGN parvo, V1 blob, V2 thin stripe, V4, and then again to inferotemporal cortex. A third stream goes from retina through LGN magno (classified due to its magnocellular cell type) to cortical processing layer 4B in area V1, V2 thick stripes, MT, and then parietal cortex. The interblob and blob streams are proposed to compute properties of visual boundaries and surfaces, respectively. Many experiments support this view (e.g., Elder and Zucker, 1998; Field et al., 1993; He and Nakayama, 1995; Lamme et al., 1999; Rogers-Ramachandran and Ramachandran, 1998).

Figure 110.1..  

Schematic diagram of processing streams in visual cortex in the macaque monkey brain. Icons indicate the response selectivities of cells at each processing stage: rainbow, wavelength selectivity; angle symbol, orientation selectivity; spectacles, binocular selectivity; right-pointing arrow, selectivity to motion in a prescribed direction. (Adapted with permission from DeYoe and Van Essen, 1988.)

The existence of such streams has led many scientists to conclude that our brains process perceptual qualities such as visual form, color, and motion using different independent modules. If the processing streams were independent modules, they could fully compute their particular processes on their own. In contrast, much perceptual data describe strong interactions between perceptual qualities. For example, changes in an object's perceived form or color can cause changes in its perceived motion, and vice versa, while changes in an object's perceived brightness can cause changes in its perceived depth, and vice versa (Egusa, 1983; Faubert and von Grünau, 1995; Kanizsa, 1974; Pessoa et al., 1996; Smallman and McKee, 1995). If geometrical properties like color, form, and depth are not computed independently, then what is the geometry by which we really see the world?

Many data suggest that the brain's processing streams compute complementary properties (e.g., Grossberg, 2000). Each stream's properties are related to those of a complementary stream much as a key fits its lock or two pieces of a puzzle fit together. We are all familiar with complementarity principles in physics, such as the famous Heisenberg Uncertainty Principle of quantum mechanics, which notes that precise measurement of a particle's position forces uncertainty in measuring its momentum, and vice versa. As in physics, the mechanisms that enable each stream in the brain to compute one set of properties prevent it from computing a complementary set of properties. Each stream exhibits complementary strengths and weaknesses. How, then, do these complementary properties get synthesized into a consistent behavioral experience?

Neural models clarify how interactions between these processing streams overcome their complementary deficiencies and generate behavioral properties that realize the unity of conscious experiences. Pairs of complementary streams are the functional units that interact together in order to compute unambiguous information about the world. Such interactions may be used to explain many of the ways in which perceptual qualities are known to influence each other. Thus, although analogies like a key fitting its lock are suggestive, they do not fully capture the interactive dynamism of what complementarity means in the brain.

Each stream can possess multiple processing stages. For example, in Figure 110.1, the LGN inputs to cortical areas V1, V2, V4, and then inferotemporal and parietal cortices. Why is this so? One reason is that these stages realize a process of hierarchical resolution of uncertainty. Uncertainty here means that computing one set of properties at a given stage can suppress information about a different set of properties at that stage. Uncertainty principles are also familiar in physics. In the brain, these uncertainties are overcome by using more than one processing stage to form a stream. Overcoming informational uncertainty utilizes both hierarchical interactions within each stream and parallel interactions between streams that overcome their complementary deficiencies. The computational unit is thus not a single processing stage; it is, rather, an ensemble of processing stages that interact within and between complementary processing streams.