Introduction

In this chapter the contributions of two multimodal brain areas, the orbitofrontal cortex and amygdala, to affective processing are evaluated in the light of evidence from human neuroimaging. Neuroimaging studies are reviewed that are consistent with at least two functions for these regions: first, in representing the affective value of stimuli in different modalities, and second, in stimulus-reinforcer learning, where associations are learned between arbitrary neutral stimuli and reinforcing stimuli (with affective value). Although processing in these regions can be confined to a single modality, in many cases the stimulus representations can be considered to be multimodal, and stimulus-reinforcer learning is usually, though not exclusively, concerned with cross-modal associations.

The orbitofrontal cortex is located on the ventral surface of the frontal lobes and may be defined as the area that receives afferent input from the mediodorsal nucleus of the thalamus (Fuster, 1997). The amygdala is a almond-shaped structure in the anterior portion of the medial temporal lobes, adjacent and anterior to the hippocampus. Both of these regions can be considered to be multimodal association areas, as they receive inputs from each sensory modality as well as from polymodal areas in the temporal lobe (Amaral & Price, 1984; Cavada, Company, Tejedor, Cruz Rizzolo, & Reinoso Suarez, 2000). There is considerable evidence from animal studies and human lesion studies to implicate the orbitofrontal cortex and amygdala in several functions. First, these regions appear to be important in representing the affective value of stimuli in many different sensory modalities (Rolls, 1999, 2000). Evidence from single-neuron neurophysiology studies (see Chap. 12) indicates that in nonhuman primates, the reward value of taste, olfactory, and visual stimuli is represented in orbitofrontal cortex, as neurons in this region respond to the taste and/or odor and/or the sight of food when an animal is hungry but decrease their responses once the animal has eaten that food to satiety (Rolls, Sienkiewicz, & Yaxley, 1989). Neurons in the primate orbitofrontal cortex have been found to code for the relative preference of a reward in that they gradually reduce their responses to a food consumed to satiety as a preference for it relative to other foods, to which the neurons continue to respond, diminishes (Rolls, 2002; Rolls et al., 1989), and respond more to a preferred reinforcer presented in a block of trials with a less preferred reinforcer (Tremblay & Schultz, 1999). Crossed unilateral lesions of the orbitofrontal cortex and amygdala impair the ability of an animal to alter its goal-directed responses toward a reward, once the value of that reward has been altered by feeding the animal to satiety on that reward (Baxter, Parker, Lindner, Izquierdo, & Murray, 2000).

A second function in which the orbitofrontal cortex is involved is that of learning to predict which events or objects in the environment are associated with reinforcement and which are not (Everitt et al., 1999; LeDoux, 1995; Rolls, 1990, 1999, 2000). Evidence for this function comes from single-neuron neurophysiological studies in nonhuman primates in which neurons in orbitofrontal cortex respond to a stimulus that has been associated with a reward or punishment but stop responding to that stimulus following a change in the reinforcement contingencies (Thorpe, Rolls, & Madison, 1983). This is an example of stimulus-reinforcer association learning in which the previously neutral stimulus may be a visual or olfactory stimulus and the primary (unlearned or innate) reinforcer may be a stimulus such as taste or touch. This learning is an example of stimulus-stimulus association learning, and typically the stimuli are in different sensory modalities. The process thus describes one way in which cross-modal neuronal responses are built in the brain. Neurons in the orbitofrontal cortex have been found to respond to cue stimuli that are predictive of subsequent reinforcement, or during a delay period in which a reward is expected (Hikosaka & Watanabe, 2000; Schoenbaum, Chiba, & Gallagher, 1998; Schultz, Tremblay, & Hollerman, 2000). Humans and nonhuman primates with orbitofrontal cortex lesions have difficulty reversing their choice of stimulus when a stimulus that was previously associated with a reward is no longer rewarded (Dias, Robbins, & Roberts, 1996; Iversen & Mishkin, 1970; Rolls, Hornak, Wade, & McGrath, 1994).

The amygdala has also been found to be involved in learning to predict subsequent reinforcement. Lesions of the amygdala abolish fear-conditioned responses to the presentation of a cue that is predictive of a subsequent aversive event (Davis, 2000). Lesions of the amygdala also impair stimulus-reward learning in rats (Everitt et al., 1999; Parkinson, Robbins, & Everitt, 2000). Neurons in the rat amygdala have been found to respond during expectation of rewards and punishments (Schoenbaum, Chiba, & Gallagher, 1999). In relation to multisensory integration, the underlying hypothesis is that the orbitofrontal cortex and amygdala are involved in implementing a particular form of cross-modal association learning in which learned associations are formed between a stimulus in one sensory modality and a primary reinforcer (a stimulus with innate affective value) in another (or the same) sensory modality (Rolls, 1990, 2000).

In this chapter, evidence from human neuroimaging of a role for the orbitofrontal cortex and amygdala in representing the affective value of stimuli in different sensory modalities, as well as in stimulus-reinforcer association learning and reversal, is presented. The first part of the chapter outlines evidence that these regions are involved in unimodal processing for several different sensory modalities, with a representation of affective or reinforcing value in each case. The involvement of these regions in multimodal processing is then discussed. Before considering this evidence, we will first discuss the imaging methods used in these studies.