Introduction

One of the classic roles attributed to multisensory integration is that of producing a unitary percept of objects. We can recognize a spoon not only by looking at it but also by touching it when it is out of view. We can recognize a species of bird by seeing it, by using visual information about its size, shape, and colors, and also by hearing its song. We can guess that a bee is approaching one side of our face by seeing it enter one side of our visual field or by hearing its buzzing near one of our ears.

Another fundamental role of multisensory integration is to help us recognize and understand what other individuals are doing. Even without seeing a person pouring water into a glass, we can recognize this action from the sound made by the flowing liquid.

In addition, action execution depends on multisensory integration. Indeed, in our daily life most of the actions we perform rely on sensory information, and in order to act appropriately, we often have to process in parallel information arriving via more than one sensory modality. The act of kicking a ball, for example, requires the integration of visual, proprioceptive, and tactile modalities. Writing is another example of an action that, to be accomplished accurately, requires the integration of visual, proprioceptive, and tactile information.

Thus, the retrieval in our brain of the representation of a given object, individual, or action greatly benefits from information arriving through different and multiple sensory modalities. Where and how are the different sensory modalities integrated?

According to the classic view, the cerebral cortex consists of sensory, motor, and associative areas that are functionally separate from one another. In particular, it has been assumed that perception constitutes the highest level in the elaboration of sensory inputs through several serial steps. According to this view, the task of associative areas is to integrate several modalities, producing a final percept, such as that of space or that of objects. The content of perception is then provided to motor areas to drive movement execution. The basis of this view is a serial model of the processes that occur in the cerebral cortex, in which the role of the motor cortices is to deal with already elaborated sensory information, thus playing little or no role in its elaboration.

A more recent conceptualization of the relationship between action and perception limns a dichotomy between a perceptuocognitive brain, which basically relies on visual information elaborated in the ventral stream, and an executive brain, which exploits the visual information elaborated in the dorsal stream for the on-line control of action (see, e.g., Goodale, 2000; for a somewhat different view, see Martin, Ungerleider, & Haxby, 2000).

Recent empirical evidence has contributed to a change in these views by emphasizing the role of the motor system not only in the control of action but also in the perceptuocognitive domain. Neuroanatomical data show that a basic architectural feature of the cerebral cortex is the presence of multiple parallel corticocortical circuits, each involving areas tightly linked with one another. Many of these circuits consist of specific parietal and frontal areas linked by reciprocal connections (for a review, see Rizzolatti & Luppino, 2001; Rizzolatti, Luppino, & Matelli, 1998). In these circuits, sensory inputs are transformed in order to accomplish not only motor but also cognitive tasks, such as space perception and action understanding.

Neurophysiological data show that the neurons of motor areas not only discharge during a monkey's active movements but can also be driven by different types of sensory stimuli, such as tactile, visual, or auditory stimuli, previously considered the typical inputs for driving neurons in the posterior part of the cerebral cortex. Particularly interesting is the demonstration of polymodal neurons in the premotor cortex. Other functional data provide evidence that the parietal cortex contains not only neurons activated by several sensory inputs but also neurons that discharge during active movements performed with different effectors. Therefore, it is possible to conclude that parietal areas are endowed with motor properties and motor areas are endowed with uni- or polymodal sensory properties (Andersen, Snyder, Bradley, & Xing, 1997; Colby & Goldberg, 1999; Hyvärinen, 1982; Mountcastle, 1995; Mountcastle, Lynch, Georgopoulos, Sakata, & Acuna, 1975; Rizzolatti, Fogassi, & Gallese, 1997, 2002; Sakata & Taira, 1994).

In this chapter we will show that multisensory integration is a pervasive feature of parietofrontal centers involved in sensorimotor planning and control. We will present and discuss three parallel parietofrontal circuits: those involved in sensorimotor transformations for action, those involved in space perception, and those involved in action understanding. We propose that motor representations play a major role by providing a binding key to multisensory integration.