Introduction

The visual and auditory systems are the exteroceptive senses that provide information about objects and events in the outside world. Both are involved in identifying and localizing stimuli in space. Often, an object may simultaneously emit both visual and acoustic signals, in which case the brain should decide that the stimuli indeed emanated from the same object. However, for that to happen, the sensory signals need to be integrated. Multisensory integration is of paramount importance for any nervous system coping with a multitude of potential stimuli that simultaneously compete for attention in the natural world. To process this information successfully, the brain must weed out irrelevant stimuli, resolve potential ambiguities in the sensory inputs arising from blurred, partially occluded, or noisy signals, and select a unique target from all the potential possibilities. The result of this process may be the generation of a rapid orienting response, allowing further detailed analysis with foveal vision. While programming or executing the movement, however, objects may change their position or their multimodal attributes, or new stimuli may appear elsewhere in the environment. The need for continuous updating of the multisensory world poses a major challenge to the brain. A crucial neural structure involved in multisensory-evoked orienting behavior is the midbrain superior colliculus (SC).

In this chapter, we focus on the integration of auditory and visual stimuli when evoking a rapid goal-directed orienting response of the eyes and head (a so-called gaze saccade, i.e., eye in space). We first describe the relevant sensorimotor response properties of SC cells in awake, behaving rhesus monkeys, and some of the multisensory characteristics of these neurons. We then describe the problems underlying auditory-visual orienting behavior, and present and discuss recent experimental results obtained in human subjects.