Introduction

The brain is an almost real-time processor that receives a constant flow of sensory information through several parallel sensory channels, binds the information from different senses, compares the stimuli with previous experience and current goals, and produces motor output to best fit the current circumstances. The sensory-specific cortices and the motor cortices have been studied in detail even in humans, whereas the processes that integrate various senses with each other as well as with the motor output are still largely unexplored. Given the technical challenges, this lack of knowledge is hardly surprising: multisensory processes involve several brain areas with widespread serial and parallel connections and with rapidly changing weights of activity in these neural networks (see, e.g., Goldman-Rakic, 1995; McClelland, Rumelhart, & Clinton, 1986; Mesulam, 1998). To unravel the neural mechanisms contributing to multisensory processing it is necessary to follow the spread of activations and interactions between various brain areas with high temporal and spatial accuracy.

Each research tool excels in revealing specific aspects of brain functions. Functional brain imaging methods based on changes in metabolism, energy consumption, or blood flow, such as positron emission tomography (PET) or functional magnetic resonance imaging (fMRI), have excellent spatial accuracy but poor temporal resolution because of the slow time course of hemodynamic and metabolic changes. Electrophysiological measures, on the other hand, have excellent temporal resolution, but accurate identification of the involved brain areas (source configurations) can be complicated because of the nonunique electromagnetic inverse problem. However, magnetoencephalography (MEG) offers several advantages in studying multisensory processing mechanisms of the human brain, allowing characterization of large-scale neurocognitive cortical networks both in time and in space. The hitherto small overlap in experimental designs of multisensory MEG versus EEG studies discourages extensive comparison across multisensory MEG and EEG results; the reader is therefore referred to other chapters in this handbook for multisensory EEG studies.

This chapter begins with a short description of the MEG method, followed by a review of results obtained by using MEG to evaluate multisensory brain functions. We have divided multisensory processing in two main categories. Real-time processing refers to integration of different sensory modalities from the constant flow of sensory information that occurs almost automatically, and in which the stimuli are naturally associated through spatiotemporal coincidence. Learning-dependent multisensory processing refers to situations in which association between different stimuli requires extensive previous experience, and the stimulus combinations can be entirely arbitrary; such processes are essential for high-level human cognition. The chapter concludes with a discussion of the plasticity of the brain from a multisensory point of view.