| |
Abstract:
The present study investigates predictive
processing during the incremental integration of visual scenes
and concurrent auditory input, using the "visual-world"
eye-tracking paradigm (e.g., Cooper, 1974; Tanenhaus et al.,
1995). Altmann & Kamide (1999) found that selectional
restrictions associated with verbs (e.g., 'eat') can be used to
predict the semantic class of forthcoming Themes (e.g., as
edible). The prediction was evidenced by anticipatory eye
movements to the appropriate Theme object (a cake) prior to the
onset of the post-verbal referring expression ('the cake').
However, the English study did not fully demonstrate that syntax
plays a role in the predictive process --- one could argue that
the anticipatory eye-movements merely reflected semantic
integration of the visual context with the (semantic
interpretation of the) sentence fragment heard thus far.
German provides an opportunity to explore whether
there is a purely syntactic component to the predictive process
by virtue of its case marking system. In principle,
case-marking on the sentence-initial noun phrase, coupled with
information about the verb, could be used to predict which object
in a concurrent visual scene might most plausibly be referred to
post-verbally.
In the main study we shall report, the following
sentential conditions were presented concurrently with a visual
scene depicting a hare, a cabbage, a fox, and a tree
(distractor):
|
(1)
|
Der Hase frißt gleich den
Kohl.
The [nom] hare eats in-a-moment the [acc] cabbage.
|
|
(2)
|
Den Hasen frißt gleich der
Fuchs.
The [acc] hare eats in-a-moment the [nom] fox.
|
The fact that the hare is nominative (most likely
the Agent) in (1) and accusative (most likely the Patient) in (2)
would suggest, in combination with the verb 'frißt' (eats),
that the cabbage in (1) and the fox in (2) are the objects which
will most plausibly be referred to post-verbally. Thus, if
syntactic (case-marking) information and the semantic information
associated with the verb interact during the prediction of the
post-verbal referring expression, the plausible objects (cabbage
and fox in (1) and (2), respectively) will be looked at more
often than the implausible objects (fox and cabbage in 1 and 2
respectively). Moreover, we should observe such a pattern
before the onset of the post-verbal noun phrase. This was
exactly what the experiment revealed (the relevant effects were
reliable before the onset of the determiner of NP2). We
conclude that anticipation during sentence processing does not
rely solely on semantic processing, but on syntactic processes
also. Overall, the results indicate the rapid integration
of the two different types of information (syntactic and
semantic) during the incremental prediction process.
References
Altmann, G. T. M., & Kamide, Y. (1999).
Incremental interpretation at verbs: Restricting the domain of
subsequent reference. Cognition, 73, 247-264.
Cooper, R. M. (1974). The control of eye
fixation by the meaning of spoken language: A new methodology for
the real-time investigation of speech perception, memory, and
language processing. Cognitive Psychology, 6,
84-107.
Tanenhaus, M. K., Spivey-Knowlton, M. J.,
Eberhard, K. M., & Sedivy, J. C. (1995). Integration of
visual and linguistic information in spoken language
comprehension. Science, 268(5217), 1632-1634.
|
