| |
Abstract:
The McGurk effect occurs when mismatched auditory and visual
spoken syllables produce speech percepts different from either of
the respective unimodal stimuli. Perceptual evidence from McGurk
experiments has been used to hypothesize early phonetic integration
of speech information, even though experiments have also shown that
the effect occurs despite relatively long crossmodal temporal
asynchronies, contradicting the early integration hypothesis. To
investigate the time course and location of integration, we
obtained event-related potentials (ERPs) from twelve adults,
prescreened for McGurk susceptibility. Stimuli were presented in an
oddball paradigm to evoke the mismatch negativity (MMN), a
neurophysiological discrimination measure, most robustly
demonstrated with acoustic contrasts. Conditions were audiovisual
McGurk stimuli, visual-only stimuli from the McGurk condition, and
auditory stimuli corresponding to the McGurk condition percepts
(/ba/-/da/). The magnitude (area) of the MMN for the audiovisual
condition was maximal at a latency > 300ms, much later than the
maximal magnitude of the auditory MMN (approximately 260ms),
suggesting that integration occurs later than auditory phonetic
processing. Additional latency, amplitude, and dipole source
analyses revealed similarities and differences between the
auditory, visual, and audiovisual conditions. Results support an
audiovisual integration neural network that is at least partly
distinct from and operates at a longer latency than unimodal
networks. (NSF 9996088)
|
