From Towards a Science of Consciousness 3 Section 4: Vision and Consciousness -- Introduction CogNet Proceedings
Contrasting with naive conceptions of perception as a pure bottom-up process, the idea has been proposed by von Helmoltz that perception results from unconscious inductive inferences (Gregory 1987). Although physiological studies of the visual system have long been focused on how visual images are constructed through hierarchically organized stages of processing, the same idea of a dialogue between bottom-up and top-down processes is now being applied to the understanding of vision (Salin and Bullier 1995, Thorpe et al. 1996). This two-way description of vision and perception in general is also widely acknowledged by psychologists (e.g., Gregory 1987, Cave and Kosslyn 1989) and philosophers (Dretske 1990, Jacob 1985), so much so that the idea that "there is no such thing as immaculate perception" has been defended (Kosslyn and Sussman 1995). The most cited experimental evidence for the implication of descending influences on perception is the case of ambiguous figures, for which perception can alternate between two possible interpretations of the visual input, even though the memorized image can be subjected to other interpretation (Peterson et al. 1994). Visual illusions are also often considered as a clear example of the interpretation (and contamination) of retinal information involved in perception (Dretske 1990, Jacob 1985, Gregory 1987, Meini 1996). In their attempt to rule out the possibility for "immaculate perception," Kosslyn and Sussman (1990) review evidence for the use of imagery in perception suggesting that a match is being created between descending expectations and ascending signals. Then they present possible anatomical substrate of descending feedback from higher visual centers, and consider possible ways to transform an internal image so as to match the peripheral retinal image. This latter issue leads them to consider the strong link between this mental transformation and sensori-motor processing. This chapter will consider these and other observations as a way to demonstrate that instances of immaculate perception can be precisely found in the field of action.
The current knowledge of visual processes does not allow to challenge the idea that most instances of perception involve downward projections from mental images. This is particularly true for high-level perception, for example when subjects are required to identify a given shape or letter. Indeed many of the examples presented in Kosslyn and Sussman's paper (1995) deal with pattern recognition. As stressed in their introduction, it is especially at the level of on-line input processing that the idea of immaculate perception can be questioned. The best examples of on-line processing of visual input are found in the field of action, because simple actions can be the locus of extremely fast processing of visual changes. Just as is the case for the "role of imagery in recognition" (Kosslyn and Sussman 1995), the role of motor representations "may be counter-intuitive because we are not aware of generating" them when we perform an action (c.f. Jeannerod 1994). But examples of specific motor representations can be found from a variety of studies ranging from neuropsychology to motor psychophysics (reviews: Rossetti 1998, Milner, this book, Goodale, this book).
The first example can be taken from the Prablanc, Pélisson, and Goodale study (1986), in which a better pointing accuracy was found when the target remained in view during the entire movement time than when it was removed shortly after movement initiation. This result has been considered by Kosslyn and Sussman as a demonstration that "a tight linkage exists between what one expects to see and an error-correction motor output system." The same authors (Goodale et al. 1986, Pélisson et al. 1986) also studied the effect of unexpectedly displacing the target during the eye saccade toward it, so that subjects were never aware of this target jump. Strikingly however, subjects corrected their movement and reached the second location of the target without increasing their movement time. In addition, they were not aware of their own arm trajectory change (Pélisson et al. 1986). This result clearly indicates that a discrepancy can be found between the on-line processing of the visual information used for driving the hand toward its target and the subjects expectation about the target location. In addition, the same error-correction mechanism is at work when the target jump can be consciously detected because it is not synchronized with the saccade (e.g., Castiello et al. 1991, Komilis et al. 199?). Although it would certainly be misleading to assume that all brain inferences are associated with conscious awareness (c.f. supra), it is difficult to conceive how the processing of an unexpected target perturbation would benefit from top-down imagery about where the target should have jumped. It may rather be argued that the speed of the motor reactions to a visual perturbation may preclude the influence of top-down mechanism in these mechanisms.
An even stronger demonstration of this discrepancy is obtained when the subject is required to stop his movement whenever the target jumps (Pisella et al. 1998). Under this particular condition, fast movements are found to reach the secondary location of the target, whereas they should have been interrupted (Rossetti and Pisella 1998). This result suggests not only that a motor representation of the target is used to drive the hand toward a new position despite there is no expectation of the jump (Pélisson et al. 1986), but also that this motor representation can be stronger than the expectation, so as to reach even a "prohibited" target. As a conclusion to these observations, it may be emphasized that the temporal issue is crucial here. When movements are slow enough, the discrepancy between the subject's expectation and the automatic motor corrections carried out by the hand can be resolved before the action ends, even though the motor reaction is initiated prior to awareness of the target perturbation (e.g., Castiello et al. 1991). This agreement can be achieved also because the detection of the unexpected event does not imply further modification of the ongoing corrective action. In the case of faster movements, and the unexpected jump being associated with an interruption of the automatic behavior, the subjects reached the prohibited target before to become apt to control their movement (Rossetti and Pisella 1998). In this particular case a clear dissociation can be observed between the fast motor representation automatically driving the hand toward a goal and the slower control processes.
Examples of pure spatial (re)presentations (not contaminated by higher level cognitive representation) can be observed in brain-damaged patients. Blindsight and Numbsense provide examples of the complete loss of sensory experience in the visual or in the somesthetic modality, where actions can still be performed toward undetected target objects (review in Rossetti 1998). Blindsight patients may be able to gaze or to point at unseen visual targets (e.g., Pöppel et al. 1973, Weiskrantz et al. 1974, Perenin and Jeannerod 1975), or even to significantly size their grasp or orient their hand while reaching to an object (Perenin and Rossetti 1996). Blindfolded numbsense patients are able to point to unfelt tactile stimuli applied to their arm (Paillard et al. 1987, Rossetti et al. 1995) and to point to unfelt proprioceptive targets (Rossetti et al. 1995) with above-chance performance. In the same way, patients with visual agnosia exhibit good goal-directed motor performances, whereas they may remain unable to describe any feature of the target object verbally (see Milner, this book). These arguments are consistent with the idea that pure motor representations can be spared in patients with lesion of higher-level perceptual systems. The example of patient J. A. is presented in figure 12.1. This patient presented a complete loss of all somatosensory processing on the left half of his whole body when he was tested clinically for light touch, deep pressure, moving tactile stimulation, pain, warm and cold, vibration, segment position, passive movement, etc. When blindfolded and required to guess verbally the locus of tactile stimuli delivered to his forearm and hand, he performed at chance level. Figure 12.1 shows his significant ability to point at stimuli delivered to his right arm despite he could not otherwise indicate where the stimulus was applied.
A crucial argument in this debate about specific motor representations has been brought by investigations of motor responses to visual illusions (see also Goodale, this book). For example, Gentilucci et al. (1996) required subjects to point from one end to the other end of Müller-Lyer lines, in order to investigate whether subjects would produce longer movements for the open configuration of the line (>---<, which appears longer) as compared to the closed configuration (<--->, which appears shorter). Although the perceptual effect of Müller-Lyer illusion can be estimated about 20 percent of the actual length (Rossetti 1998), the motor performance was not biased by more than 2-3 percent. These results are consistent with previous results that demonstrated that the frame of reference used by the action system was different from that used by conscious perception (Bridgeman et al. 1997). The action system would compute the position of targets in an egocentric reference frame, whereas perception would locate objects relative to the surrounding frame or objects (see also Goodale and Milner 1992, Jeannerod and Rossetti 1993, Milner and Goodale 1995). These results also suggest that despite the very strong and reliable effect of this illusion, that has been used to argue for the extensive interpretation of visual input by cognitive processes, the motor representation of the line length was only very marginally contaminated by such interpretation (especially if one considers that a component of Müller-Lyer illusion can be explained by retinal factors, as mentioned by Gentilucci et al. 1996). Interestingly, the effect of the illusion on pointing was dramatically increased when a delay was introduced between the line presentation and the motor response (Gentilucci et al. 1996). A very similar effect of delay on pointing actions as shown previously has been described by Rossetti and Régnier (1995) for proprioceptive targets and by Bridgeman (1997, 1998) for visual targets. In these two studies the introduction of a memory delay between the target presentation and the response induced a change in the reference frame used by the action system to located the target. Analyses of the delayed pointing errors revealed that they resulted from the computation of the goal with respect to the surrounding information. These results suggest that motor representations are short-lived, and that the information processed by higher cognitive representation is being feeded into the motor system when the delay exceeds its limited memory capacity, therefore contaminating the outcome of the action.
This reliable effect of delay is compatible with the idea that ". . . imagery is a bridge between perception and motor control" (Kosslyn and Sussman 1995: 1040) (see Rossetti and Procyk 1997). When the life time of motor representations is too short to allow them to participate in the action, the motor response has to rely on spatial memory. Just as ambiguous images benefit from visual imagery, the spatial memory used to guide a delayed action can be contaminated by higher-level interpretation of the spatial relationship between the target and its surrounding. Figure 12.2 exemplifies this specific influence of imagery on a delayed action: immediate pointings are not influenced by the spatial context of the experimental session, whereas delayed pointing are strongly influenced by the integration in time of the successive target location used in the same session (Rossetti and Régnier 1995, Rossetti et al. 1996).
Is it possible to conclude that motor representations provide one example of immaculate perception? It can be argued that motor representations are not innate and benefit from early-childhood learning, such that they could not be considered as immaculate perception. One interesting aspect of this learning, however, is that what is learned by the motor system results from a direct confrontation with surrounding objects. Motor responses are necessarily constrained by the metric properties of the physical environment, whereas perception is not (see also Bridgeman 1998). If there is a perfect metric correspondence between the space representation used to drive an action and the physical space of the action goal, as supported by the data presented here, then motor representations are clearly pure from distorting brain inferences, and can be considered as an instance of immaculate perception. Their very short-lived feature may be the characteristic that prevent them from being influenced by visual imagery.
Bridgeman, B. 1998. Interactions between vision for perception and vision for behavior. In Y. Rossetti and A. Revonsuo (Eds.). Beyond dissociation: interactions between dissociated implicit and explicit processing. In press.
Bridgeman, B., S. Peery, and S. Anand. 1997. Interaction of cognitive and sensorimotor maps of visual space. In Percept Psychophys 59(3):456-469.
Castiello, U. 1991. Temporal dissociation of motor responses and subjective awareness. A study in normal subjects. In Brain 114:2639-2655.
Cave, K. R., and S. M. Kosslyn. 1989. Varieties of size-specific visual selection. In J. Exp. Psychol. [Gen] 118:148-164.
Dretske, F. 1990. Seeing, believing, and knowing; in D.N. Osherson, S. M. Kosslyn, and J. M. Hollerbach (Eds.): An Invitation to Cognitive Science. MIT Press, 129-148.
Gentilucci, M., Chieffi, S., Daprati, E., and Toni, I. 1996. Visual illusion and action. Neuropsychologia 34:369-376.
Goodale, M. A., and A. D. Milner. 1992. Separate visual pathways for perception and action. In Trends in Neurosc. 15:20-25.
Goodale, M. A., D. Pèlisson, and C. Prablanc. 1986. Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. In Nature 320(6064):748-850.
Gregory, R. L. 1987. The Oxford companion to the mind. Oxford: Oxford University Press. Jacob, P. 1985. Voit-on ce que l'on croit? Philosophie 7:53-82.
Jeannerod, M., and Y. Rossetti, Y. 1993. Visuomotor coordination as a dissociable function: experimental and clinical evidence; in C. Kennard (Ed): Visual perceptual defects. London, Ballière Tindall, pp. 439-460.
Jeannerod, M. 1994. The representating brain: Neural correlates of motor intention and imagery. In Behavioral and Brain Sciences 17(2):187-245.
Komilis, E., D. Pelisson, and D. Prablanc. 1993. Error processing in pointing at randomly feedback-induced double-step stimuli. In Journal of Motor Behavior 25 (4):299-308.
Kosslyn, S. M., and A. L. Sussman. 1995. Roles of imagery in perception: or, there is no such thing as immaculate perception; in M. S. Gazzaniga. (Ed): The Cognitive Neurosciences. Cambridge, Massachussetts, MIT Press, pp. 1035-1042.
Meini, C. 1996. La frontière entre percept et concept: la théorie de Fred Dretzke à l'épreuve des illusions d'optique. In Cognito, (6), 27-38.
Milner, A. D., and M. A. Goodale, M. A. 1995. The visual brain in action (Oxford Psychology Series 27). Oxford: Oxford University Press. 248 pp.
Pélisson, D., C. Prablanc, M.A. Goodale, and M. Jeannerod. 1986. Visual control of reaching movements without vision of the limb. II. evidence of fast unconscious processes correcting the trajectory of the hand to the final position of a double-step stimulus. In Experimental Brain Research 62:303-311.
Perenin, M-T., Y. and Rossetti. 1996. Grasping in an hemianopic field. Another instance of dissociation between perception and action. In Neuroreport 7:793-897.
Perenin, M-T., and M. Jeannerod. 1975. Residual vision in cortically blind hemifields. In Neuropsychologia, 13:1-8.
Peterson, M. A., J. F. Kihlstrom, P. M. Rose, and M. L. Glisky. 1992. Mental images can be ambiguous: reconstruals and reference-frame reversals. In Memory and Cognition 20(2):107-123.
Pisella, L., M. Arzi, and Y. Rossetti. 1998. The timing of color and location processing in the motor context. In Experimental Brain Research In press.
Pöppel, E., R. Held, and D. Frost. 1973. Residual visual function after brain wounds involving central visual pathways in man. In Nature 243:295-296.
Prablanc, C., D. Pelisson, and M. A. Goodale. 1986. Visual control of reaching movements without vision of the limb. I. Role of retinal feedback of target position in guiding the hand. In Exp Brain Res. 62(2):293-302.
Rossetti, Y., F. Gaunet, and C. Thinus-Blanc. 1996. Early visual experience affects memorization and spatial representation of proprioceptive targets. In Neuroreport 7(6):1219-1223.
Rossetti, Y., and C. Régnier. 1995. Representations in action: pointing to a target with various representations; in B. G. Bardy, R. J. Bootsma, and Y. Guiard (Eds.): Studies in Perception and Action III. Mahwah, N.J., Lawrence Erlbaum Associates, Inc., pp. 233-236.
Rossetti, Y., G. Rode, and D. Boisson. 1995. Implicit processing of somaesthetic information: a dissociation between where and how? In Neuroreport 6:506-510.
Rossetti, Y. 1998. Short-lived motor representation of space in brain-damaged and healthy subjects. In Consciousness and Cognition, in press.
Rossetti, Y., and L. Pisella. 1998. Temporal asynchrony between sensory and between motor components of a visuo-motor response: a time-grounded dissociation between implicit and explicit processing. Tucson III, abstract no. 217.
Rossetti, Y., and E. Procyk. 1997. What memory is for action: the gap between percepts and concepts. In Behavioral and Brain Sciences<.I> 20(1):34-36.
Salin, P. A., and J. Bullier. 1995. Corticocortical connections in the visual system: structure and function. In Phyiol. Reviews 75:107-154.
Thorpe, S., D. Fize, and C. Marlot. 1996. Speed of processing in the human visual system. In Nature 381:520-522.
Weiskrantz, L., E. R. Warrington, M.D. Sanders, and J. C. Marshall. 1974. Visual capacity in the hemianopic field following a restricted occipital ablation. In Brain 97:709-828.