| |
Abstract:
Connectionist models of sentence processing are appealing
because of their ability to fit quantitative data. However, it has
not been clear up to now by what principles they can handle the
complex structural phenomena that linguistic theories of syntax
reveal (e.g., Fodor & McLaughlin, 1990).
This paper describes a method of designing (as opposed to
learning) the representations of a special kind of connectionist
network called the Dynamical Automaton (DA). DAs can mimic the
representational properties of Context Free Grammars and can thus
be used for encoding many of the phrasal phenomena of natural
languages. The essential idea is to use fractal sets which are
self-similar at arbitrarily small scales to encode the recursive
structure of phrases in the bounded representation space of a set
of neurons. Once programmed, a DA exhibits similar behavior to
Elman (1991)'s Simple Recurrent Network, but it has the advantage
that its representational principles are transparent.
DAs provide a natural account of memory load phenomena like the
contrast in difficulty between subject and object relative clauses.
The essential idea combines Gibson (1998)'s insight that memory and
integration costs are proportional to the length of time a
partially completed constituent is stored with McRae,
Spivey-Knowlton, and Tanenhaus (1998)'s dynamical processing
mechanism. Each unfinished constituent that is stored in memory
requires using fractal structure at an exponentially smaller scale.
Therefore noise in the neurons, which blurs smaller contrasts more
than larger ones, has a more distorting effect when there is
greater load on the memory, and the dynamical system takes longer
to recover when the stored items are eventually recalled. The
resulting reading time profiles for subject and object relative
clauses closely resemble the data of King and Just (1991) and
Gibson and Ko (in preparation). The account improves upon Gibson
(1998)'s treatment empirically and also eliminates the need for
separate integration and memory-load cost components. It achieves
comparable data coverage to MacDonald and Christiansen (1998)'s
connectionist account with the advantage of an explicit analysis of
the representational properties of the network. I suggest that
Dynamical Automata provide a suitable general framework for
combining the strengths of connectionist and symbolic models.
References
Elman, J. L. (1991). Distributed representations, simple
recurrent networks, and grammatical structure.
Machine Learning,
7, 195-225.
Fodor, J., & McLaughlin, B. P. (1990). Connectionism and the
problem of systematicity: Why Smolensky's solution doesn't work.
/I> 35, 183-204.
Gibson, E. (1998). Linguistic complexity: Locality of syntactic
dependencies.
Cognition, 68(1), 1-76.
Gibson, E., & Ko, K. (in preparation). Processing main and
embedded clauses. Department of Brain and Cognitive Sciences, MIT.
MacDonald, M. C., & Christiansen, M. H. (1998). Individual
differences without working memory: A reply to Just & Carpenter
and Waters & Caplan. Manuscript, USC.
McRae, K., Spivey-Knowlton, M. J., & Tanenhaus, M. K. (1998).
Modeling the influence of thematic fit (and other constraints) in
online sentence comprehension.
Journal of Memory and Language,
38, 283-312.
|
