Background

The mixing of inputs from left and right eyes in the visual cortex is not complete, but it does provide the basis for the physiological fusion of monocular images. There are minor species differences in this regard since the population of cells in input layers of the primate is more monocular than that of the cat (Hubel and Wiesel, 1962, 1968). Monocular pathways clearly support a two-dimensional analysis of the visual world. The image from each eye projects to regions that produce two-dimensional maps of object space. These maps are interpreted in monocular pathways. Because the two eyes have slightly different views of visual space, their respective images are displaced on the retinas. This retinal disparity is the necessary and sufficient condition for stereoscopic depth discrimination. It allows the two-dimensional projection of visual space to become a three-dimensional percept. Disparity-based stereopsis was demonstrated in early psychophysical studies (Wheatstone, 1838).

Details of the anatomical pathways that subserve binocular vision are covered in other chapters (see Chapter 37). The primary relevant physiological finding is that neurons in the striate cortex or V1 are responsive binocularly. The original demonstration of this, from studies of area 17 in the cat, was mainly by procedures in which tests were conducted alternately between left and right eyes. In other words, conclusions about binocular function were made from monocular tests. This is problematic because, as shown below, monocular tests can produce faulty assessments of the degree of binocularity of a cell. To be fair, the landmark study of binocular interaction in the cat (Hubel and Wiesel, 1962) included tests of both eyes simultaneously. It was also noted that some cells responded only when both eyes were activated simultaneously. However, subsequent to the early study, most experiments in which binocularity has been assessed have used only monocular tests (see below).

One other factor should be noted in connection with the physiological study of binocular vision. In the early investigations, the suggestion was made that binocular interaction in the visual cortex could be connected to the physiological basis of stereoscopic depth discrimination (Hubel and Wiesel, 1962). However, the way in which this might be accomplished was not specified. In subsequent investigations, direct attempts were made to identify neurons that responded to stimuli positioned at different depths. The factor referred to above is that essentially all the early work was conducted under the assumption that cortical neurons are depth detectors. This is, of course, a reasonable approach, and it follows the work of a long line of investigators who were interested in trigger features as they applied to a particular animal (for example, Lettvin et al., 1959). One limitation of this approach is that a given experiment and the main results are linked to a specific interpretation. In the case of depth-detecting neurons, it is possible that the disparity selectivity of cortical cells is a result of RF organization. There may be no direct linkage to a perceptual process. It is very important to establish this link, and recent work on alert, behaving monkeys enables a firm connection to be made between RF response characteristics and behavioral performance (Cumming and Parker, 2000; DeAngelis and Newsome, 1999).

It is possible to approach the physiological study of binocular vision without the assumption that neurons in the primary visual cortex are depth detectors. In this case, the question addressed is: how do binocular cells combine inputs from right and left eyes? The approaches used to answer this question are similar to those that have been undertaken for investigations of monocular pathways. In the most general sense, the purpose is to establish how a cell integrates light within its RF. This approach offers the potential of establishing the mechanisms of monocular and binocular processing in a unified manner.

Before describing the studies in which this approach has been applied, it is useful to summarize the classical work on the physiological basis of stereoscopic depth discrimination. The first direct study was performed in anesthetized paralyzed cats (Barlow et al., 1967). Neurons in area 17 were found to be selectively responsive to different retinal disparities corresponding to spatial depth ranges from near to far distances, as depicted in Figure 48.1. Preferences were found for a large range of both crossed and uncrossed disparities. However, the reported range of disparities was three times larger for horizontal compared to vertical disparities. This has a functional advantage because the horizontal displacement of the two eyes means that this orientation is required for stereopsis. Subsequent studies did not confirm this observation (Ferster, 1981; LeVay and Voigt, 1988; Nikara et al., 1968), but some hint of this asymmetry was found in other work (Joshua and Bishop, 1970; von der Heydt et al., 1978). By analysis of phase (shape) differences between the RFs of left and right eyes (to be described below), it has been shown that there are clear disparity-processing dissimilarities between horizontal and vertical orientations (DeAngelis et al., 1991).

Figure 48.1..  

In this conventional view of disparity-selective neurons, the cat fixates a position on an arc tangent screen which traces the locus of contrast disparity. The data points represent cells tuned to relative retinal disparities corresponding to different depths in space.

Following the initial results of studies on disparity-sensitive cells in the cat's visual cortex, an attempt was made to replicate these findings in monkey striate cortex. The first result of this attempt was remarkable because the claim was made that there were no disparity-sensitive cells in V1 but there were in V2 (Hubel and Wiesel, 1970). This conclusion was wrong, of course, but it took studies in the awake, behaving monkey to sort out the issue. A series of experiments by Poggio and collaborators established conclusively that neurons in V1 and V2 are sensitive to relative changes in stereoscopic depth (Poggio and Talbot, 1981; Poggio and Fischer, 1977; Poggio et al., 1988).

The awake, behaving monkey setup is illustrated in Figure 48.2. An animal is trained to fixate a location on a screen that can be positioned at different depths. During fixation, individual cortical cells are recorded. In the initial study, four types of depth-sensitive neurons were identified: tuned excitatory, tuned inhibitory, near, and far cells. Response curves for these categories are shown in Figure 48.3. This scheme was modified subsequently (Poggio et al., 1988). However, it is reasonably clear from other work (LeVay and Voigt, 1988) that a four- or six- or n-category system is not adequate and not correct. The reasons for this inadequate classification system are explained in detail in a previous review (Freeman and Ohzawa, 1990).

Figure 48.2..  

An experimental arrangement is shown for behavioral and neurophysiological testing of a primate. The monkey views a fixation point on a stimulus display and, through a beam splitter, on a second screen. The stimulus display may be moved fore and aft to change the relative depth. RFs of cortical cells are projected onto the stimulus display. The monkey presses a key when the fixation spot is illuminated. Visual targets used to explore neural activity are projected to the monkey's eye via the beam splitter.

Figure 48.3..  

Four cortical cell types are illustrated according to their response patterns. Horizontal disparity is indicated on the abscissas, and response is represented on the ordinates. Crossed and uncrossed disparities are located to the left and right, respectively, of the zero position. Tuned excitatory and inhibitory cells respond or are suppressed, respectively, at zero disparity. Near and far cells respond to close or far distances, respectively.

In none of the early studies was an attempt made to develop a functional mechanism within which a disparity encoding and detecting system might fit. The problems with a disparity processing system based on four or six categories of cell types may be solved by the use of an encoding scheme that is scaled to the size or spatial frequency selectivity of cortical neurons. To do this, we need to analyze RF shape (phase) as well as relative position. In this way, we can determine if binocular disparity information is coded and represented in the visual cortex by phase-disparity selective neurons at different size or spatial frequency scales. The basis of this approach is detailed below.