Attentional response modulation in visual cortex

Convergent evidence from single-cell recording studies in monkeys and functional brain imaging and event-related potential (ERP) studies in humans indicates that selective attention can modulate neural processing in visual cortex. Attention may affect visual processing in several ways, such as facilitating neural processing by enhancing neural responses to attended stimuli, filtering of unwanted information by counteracting suppressive interactions induced by nearby stimuli, or biasing neural signals in favor of an attended location or stimulus attribute by increasing the baseline activity in the absence of visual input.

Attention Facilitates Visual Processing at Attended Locations

Functional brain mapping studies in the human visual cortex suggest that attentional mechanisms operate by enhancing neural responses to stimuli at attended locations, thereby facilitating information processing in favor of stimuli appearing at that location. In a typical functional brain imaging experiment, identical visual stimuli were presented simultaneously to corresponding peripheral field locations to the right and left of fixation while subjects were asked to direct attention covertly, that is, without executing eye movements to the right or the left. Directing attention to the left hemifield led to increased stimulus-evoked activity in extrastriate visual areas of the right hemisphere, whereas directed attention to the right hemifield led to increased activity in extrastriate visual areas of the left hemisphere (Heinze et al., 1994; Vandenberghe et al., 1997). Thus, responses to the stimuli were enhanced in the regions of extrastriate cortex containing the representations of the attended hemifield. In ERP studies that used a similar experimental design, it was shown that response enhancement in extrastriate cortex due to spatially directed attention may occur as early as 80 to 130 msec after stimulus onset (Heinze et al., 1994; Hillyard et al., 1998).

These results are in accordance with single-cell recording studies, which have shown that neural responses to a single stimulus presented within a neuron's receptive field (RF) are enhanced when the animal directs its attention to the RF compared to when the animal attends elsewhere. This effect, which increases with task difficulty (Spitzer and Richmond, 1991), has been demonstrated at many levels of visual processing, including areas V1 (Motter, 1993), V2 (Luck et al., 1997; Motter, 1993), V4 (Connor et al., 1997; Luck et al., 1997; McAdams and Maunsell, 1999; Moran and Desimone, 1985; Motter, 1993; Reynolds et al., 1999), MT/MST (Recanzone and Wurtz, 2000; Treue and Maunsell, 1996), and LIP (Bushnell et al., 1981; Colby et al., 1996; Gottlieb et al., 1998). Not surprisingly, in human cortex, spatial attention effects have been found at similar levels of visual processing. Furthermore, it has been demonstrated that these attention effects are retinotopically organized, that is, response enhancement is seen only in the regions containing the sensory representation of an attended stimulus within a visual area (Brefczynski and DeYoe, 1999; Tootell et al., 1998). At the same time, the activity of neurons that represent unattended locations of the visual field appears to be attenuated. The exact nature of this response attenuation is not known.

Importantly, there is increasing evidence from single-cell physiology and functional brain imaging studies that neural responses can be modulated by attention as early as in V1 (Ito and Gilbert, 1999; Martinez et al., 1999; Motter, 1993; Roelfsema et al., 1998). Attentional response modulation in V1 appears to be more variable than in extrastriate cortex and may depend critically on a number of factors including task difficulty, competition from surrounding objects, and the need to integrate context. Attentional effects are typically smaller in early areas such as V1 and V2 compared to more anterior extrastriate areas such as V4 and TEO, indicating that attentional response enhancement increases from early to later stages of visual processing (e.g., Kastner et al., 1998). This finding suggests that attentional effects in early visual areas may be caused by reactivation from higher-order extrastriate areas. Single-cell recording studies support this idea by showing that attentional effects in area TE of inferior temporal cortex have a latency of about 150 msec (Chelazzi et al., 1998), whereas attentional effects in V1 have a longer latency of about 230 msec (Roelfsema et al., 1998). However, these latency differences may as well be attributed to local computations within areas.

In summary, spatial attention effects operate by enhancing neural responses evoked by visual stimuli at an attended location and have been demonstrated at all stages of visual processing, including primary visual cortex.

Attention Facilitates Visual Processing of Attended Stimulus Attributes or Objects

Attention modulates neural responses not only to visual stimuli at attended locations, but also to attended stimulus attributes. This has been shown in experiments that compared neural activity evoked within different visual cortical areas while subjects performed a task requiring selective attention to particular features of identical visual stimuli. In one such study, Corbetta et al. (1991) found that selective attention to either shape, color, or speed enhanced activity in the regions of extrastriate visual cortex that selectively process these same attributes. Attention to shape and color led to response enhancement in regions of the posterior portion of the fusiform gyrus, including area V4. Attention to speed led to response enhancement in areas MT/MST. Other investigations have shown that attention to faces or houses led to response enhancement in the fusiform face area (FFA) or the parahippocampal place area (PPA), respectively (Woijcuilik et al., 1998). These areas are located on the midanterior portion of the fusiform gyrus and are specialized for the processing of faces, houses, and other objects. Taken together, these results support the idea that selective attention to a particular stimulus attribute modulates neural activity in those extrastriate areas that preferentially process the selected attribute.

In typical visual scenes, many different stimulus attributes are combined in a particular object. How is the object as a whole processed if only one of its many attributes is attentionally selected? O'Craven and colleagues (1999) investigated this important question by presenting their subjects with a moving face superimposed on a house stimulus or with a moving house superimposed on a face stimulus. Subjects were instructed to attend selectively either to the face, the house, or the direction of motion. The activity evoked by these complex stimuli was investigated in the FFA, the PPA, and MT/MST, that is, in those areas that are specialized for the three different attributes contained in the stimuli. When the subjects attended to the direction of motion, the activity in the FFA was larger with moving face stimuli than with moving house stimuli. A complementary result was obtained in the PPA with moving house stimuli. These studies suggest that objects as a whole rather than single attributes of objects are the units of attentional selection, even when only one single attribute is behaviorally relevant.

Attention Helps to Filter Out Unwanted Information

Thus far, selective attention has been shown to operate by enhancing neural responses to a stimulus at an attended location or to an attended stimulus attribute. However, a typical visual scene contains multiple stimuli. Recent single-cell recording studies suggest that multiple stimuli in cluttered visual scenes compete for processing resources and that selective attention biases competitive interactions. In these studies, responses to a single visual stimulus presented alone in a neuron's RF were compared with the responses to the same stimulus when a second one was presented simultaneously within the same RF (Moran and Desimone, 1985; Recanzone et al., 2000; Reynolds et al., 1999). The responses to the paired stimuli were shown to be a weighted average of the responses to the individual stimuli when presented alone. For example, if a single good stimulus elicited a high firing rate and a single poor stimulus elicited a low firing rate, the response to the paired stimuli was reduced compared to that elicited by the single good stimulus. This result indicates that two stimuli present at the same time within a neuron's RF are not processed independently, but rather interact with each other in a mutually suppressive way. This sensory suppressive interaction among multiple stimuli has been interpreted as an expression of competition for neural representation.

Single-cell recording studies have also shown that sensory suppressive interactions can be modulated by directed attention. In particular, in extrastriate areas V2 and V4, spatially directed attention to an effective stimulus within a neuron's RF eliminated the suppressive influence of a second stimulus presented within the same RF (Reynolds et al., 1999). Attentional effects were less pronounced when the second stimulus was presented outside the RF, suggesting that competition for processing resources within visual cortical areas takes place most strongly at the level of the RF. These findings imply that attention may resolve the competition among multiple stimuli by counteracting the suppressive influences of nearby stimuli, thereby enhancing information processing at the attended location. This may be an important mechanism by which attention filters out unwanted information from cluttered visual scenes (Desimone and Duncan, 1995).

Sensory suppression among multiple stimuli and its modulation by spatial attention has also been examined in the human cortex using fMRI (Kastner et al., 1998). In these studies, subjects were presented with images of colorful, complex stimuli in four nearby locations of the upper right quadrant of the visual field while they maintained fixation. Fixation was ensured by having subjects count the occurrences of Ts or Ls at fixation, an attentionally demanding task. The stimuli were presented under two different conditions, simultaneous and sequential. In the sequential condition, a single stimulus appeared in one of the four locations, then another appeared in a different location, and so on, until each of the four stimuli had been presented in the different locations. In the simultaneous condition, the same four stimuli appeared in the same four locations, but they were presented at the same time. Thus, integrated over time, the physical stimulation parameters were identical in the two presentation conditions, but sensory suppression among stimuli could take place only in the simultaneous presentation condition. It was therefore predicted that activation in the simultaneous presentation condition would be less than in the sequential presentation condition.

Activation of V1 and ventral stream extrastriate areas V2 to TEO was found under both stimulus presentation conditions compared to interleaved blank periods. Although the fMRI signal was similar in the two presentation conditions in V1 (Fig. 101.2A), the activation was reduced in the simultaneous condition compared to the sequential condition in V2, and this reduction was especially pronounced in V4 (Fig. 101.2A) and TEO, as predicted from the sensory suppression hypothesis. Importantly, the sensory suppression effect appeared to be scaled to the RF size of neurons within visual cortical areas. That is, the small RFs of neurons in V1 and V2 would encompass only a small portion of the visual display, whereas the larger RFs of neurons in V4 and TEO would encompass all four stimuli. Therefore, suppressive interactions among the stimuli within RFs could take place most effectively in these more anterior visual areas.

Figure 101.2..  

Sensory suppression and attentional modulation in human visual cortex. A, Sensory suppression in V1 and V4. As shown by the time series of fMRI signals, simultaneously presented stimuli (SIM) evoked less activity than sequentially presented stimuli (SEQ) in V4 but not in V1. This finding suggests that sensory suppressive interactions were scaled to the receptive field size of neurons in visual cortex. Presentation blocks were 18 seconds. B, Attentional modulation of sensory suppression. The sensory suppression effect in V4 was replicated in the unattended condition of this experiment when the subjects' attention was directed away from the stimulus display (unshaded time series). Spatially directed attention (gray-shaded time series) increased responses to simultaneously presented stimuli to a larger degree than to sequentially presented ones in V4. Presentation blocks were 15 seconds. (Adapted from Kastner et al., 1998).

The effects of spatially directed attention on multiple competing visual stimuli were studied in a variation of the paradigm used to examine sensory suppressive interactions among simultaneously presented stimuli, described above. In addition to the two different visual presentation conditions, sequential and simultaneous, two different attentional conditions were tested, unattended and attended. During the unattended condition, attention was directed away from the visual display by having subjects count Ts or Ls at fixation, exactly as in the original study. In the attended condition, subjects were instructed to attend covertly to the stimulus location closest to fixation in the display and to count the occurrences of one of the four stimuli, which was indicated before the scan started. Based on the results of monkey physiology studies, it was predicted that attention should reduce sensory suppression among stimuli. Thus, responses evoked by the competing, simultaneously presented stimuli should be enhanced more strongly than responses evoked by the noncompeting sequentially presented stimuli (Luck et al., 1997; Moran and Desimone, 1985; Recanzone et al., 2000; Reynolds et al., 1999).

As illustrated in Figure 101.2B for area V4, directed attention to the display enhanced responses to both the sequentially and the simultaneously presented stimuli (gray-shaded blocks). This finding confirmed the effects of attentional response enhancement shown in numerous earlier studies in monkeys and humans, as cited previously. More importantly, and in accordance with the prediction from monkey physiology studies, directed attention led to greater increases in fMRI signals to simultaneously presented stimuli than to sequentially presented stimuli. Thus, attention partially canceled out the suppressive interactions among competing stimuli. The magnitude of the attentional effect scaled with the magnitude of the suppressive interactions among stimuli, with the strongest reduction of suppression occurring in areas V4 and TEO. These findings support the idea that directed attention enhances information processing of stimuli at the attended location by counteracting suppression induced by nearby stimuli, which compete for limited processing resources. In essence, unwanted distracting information is effectively filtered out. The degree to which distracting information can be eliminated depends on the load of the target task. For example, Rees et al. (1997) demonstrated that activation in area MT evoked by distracting moving stimuli was totally abolished when subjects performed a high-load linguistic task at fixation compared to a low-load version of the task. Thus, the greater the attentional resources devoted to the target, the less the processing of irrelevant distracting stimuli.

In summary, areas at intermediate levels of visual processing such as V4 and TEO appear to be important sites for the filtering of unwanted information by counteracting competitive interactions among stimuli at the level of the RF. This notion has recently been supported by studies in a patient with an isolated V4 lesion and in monkeys with lesions of areas V4 and TEO (De Weerd et al., 1999; Gallant et al., 2000). Subjects were asked to discriminate the orientation of a grating stimulus in the absence and in the presence of surrounding distracter stimuli. Significant performance deficits were observed in the distracter-present but not in the distracter-absent condition, suggesting a deficit in the efficacy of the filtering of distracter information.

Attention Affects Neural Activity Even in the Absence of Visual Input

There is evidence that attentional top-down signals can be obtained not only by the modulation of visually driven activity, but also in the absence of any visual stimulation whatsoever. Single-cell recording studies have shown that spontaneous (baseline) firing rates were 30% to 40% higher for neurons in areas V2 and V4 when the animal was cued to attend covertly to a location within the neuron's RF before the stimulus was presented there, that is, in the absence of visual stimulation (Luck et al., 1997). A similar effect was demonstrated in dorsal stream area LIP (Colby et al., 1996). This increased baseline activity, termed the baseline shift, has been interpreted as a direct demonstration of top-down feedback, biasing neurons representing the attended location and thereby favoring stimuli that will appear there at the expense of those appearing at unattended locations.

Attention-related top-down signals in the human visual cortex in the absence of visual stimulation were studied by adding a third experimental condition to the design used to investigate sensory suppressive interactions and their modulation by attention, as described above (Kastner et al., 1999). In addition to the two visual presentation conditions, sequential and simultaneous, and the two attentional conditions, unattended and attended, an expectation period preceding the attended presentations was introduced. The expectation period, during which subjects were required to direct attention covertly to the target location and instructed to expect the occurrences of the stimulus presentations, was initiated by a marker presented briefly next to the fixation point 11 seconds before the onset of the stimuli. In this way, the effects of attention in the presence and absence of visual stimulation could be studied.

It was found that during the expectation period preceding the attended presentations, regions within visual areas with a representation of the attended location were activated. This activity was related to directing attention to the target location in the absence of visual stimulation. Notably, the increase in activity during expectation was topographically specific, inasmuch as it was only seen in areas with a spatial representation of the attended location. As illustrated for area V4 in Figure 101.3A, the fMRI signals increased during the expectation period (EXP; textured epochs in the figure), before any stimuli were present on the screen. This increase of baseline activity was followed by a further increase of activity evoked by the onset of the stimulus presentations (ATT; gray-shaded epochs in the figure). The baseline increase was found in all visual areas with a representation of the attended location. It was strongest in V4 but was also seen in early visual areas. It is noteworthy that baseline increases were found in V1, even though no significant attentional modulation of visually evoked activity was seen in this area. This dissociation suggests that different mechanisms underlie the effects of attention on visually evoked activity and on baseline activity. Importantly, the increase in baseline activity in V1 has also been found to depend on the expected task difficulty. Ress and colleagues (2000) showed that increases in baseline activity were stronger when subjects expected a visual pattern that was difficult to discriminate compared to a pattern that was easy to discriminate. In areas that preferentially process a particular stimulus feature (e.g., color or motion), increases in baseline activity were shown to be stronger during the expectation of a preferred compared to a nonpreferred stimulus feature (Chawla et al., 1999).

Figure 101.3..  

Increases of baseline activity in the absence of visual stimulation. A, Time series of fMRI signals in V4. Directing attention to a peripheral target location in the absence of visual stimulation led to an increase of baseline activity (EXP; textured blocks), which was followed by a further increase after the onset of the stimuli (ATT; gray-shaded blocks). Baseline increases were found in both striate and extrastriate visual cortex. B, Time series of fMRI signals in FEF. Directing attention to the peripheral target location in the absence of visual stimulation led to a stronger increase in baseline activity than in visual cortex; the further increase of activity after the onset of the stimuli was not significant. Sustained activity was seen in a distributed network of areas outside the visual cortex, including SPL, FEF, and SEF, suggesting that these areas may provide the source for the attentional top-down signals seen in visual cortex. (Adapted from Kastner et al., 1999).

The baseline increases found in human visual cortex may be subserved by increases in the spontaneous firing rate similar to those found in single-cell recording studies (Colby et al., 1996; Luck et al., 1997) but summed over large populations of neurons. The increases evoked by directing attention to a target location in anticipation of a behaviorally relevant stimulus at that attended location are thus likely to reflect a top-down feedback bias in favor of the attended location in human visual cortex.