From Towards a Science of Consciousness 3 Section 4: Vision and Consciousness -- Introduction CogNet Proceedings
Part of the "Grand Illusion" of complete and direct perception is the transparency of our eye-movements. We simply don't notice them. The visual information from the retina that supports visual consciousness is sampled discontinuously in the brief fixations that normally occur two or three times per second. Our eyes make abrupt movements called saccades nearly every second of our waking lives. These eye "jumps" connect the individual fixations we make, gathering visual information that underlies our actions and our perceptual experience.
Although often crucial to the successful recovery of visual information in which we are consciously interested, our eye movements are, by and large, unconscious actions. They may be said to represent an aspect of the information-gathering control structures postulated by Gibson (1966), though they are not, themselves, part of our awareness. Given the importance of eye-movements for the retrieval of visual information from the environment, the question arises whether the eye-movement control system is capable of implicit learning, or learning without awareness. The present studies were undertaken to investigate this possibility. Can eye-movement patterns show learned sensitivity to environmental regularities of which we are not consciously aware?
In part, the motivation of these studies derived from evidence that visual consciousness often goes beyond the information available to visual cognition. For example, a visual texture can appear to be seen in clear detail ñ each element clearly represented. Yet studies of texture adaptation (e.g., Durgin and Proffitt 1996) indicate that our perceptual experience is based on processes of "biological image compression." This means that the amount of information actually available in cognition is vastly less than would be required to completely specify the detail that seems to us to be evident in our conscious experience. Similarly, recent interest in short-term perceptual memory has been fueled by the apparent discrepancy between the amount of information that seems to be present in consciousness and our insensitivity to fairly large alterations in the content of our environment from moment to moment (e.g., Grimes 1996, Rensink et al. 1997). In what ways might the sophisticated control of eye-movements help to support the "Grand illusion" of complete perception? Part of the motivation for this research came from the literatures on implicit learning (cf. Berry and Dienes 1993, Reber 1993, for reviews). If we are not really "seeing" all that we think we are seeing, might there nonetheless be (unconscious) information available to guide the visual-information-acquisition systems? Might not the whole nature of visual-information acquisition actually involve fairly complicated, yet unconscious, smart routines for guiding the control of eye-movements.
To test this idea, I have developed a paradigm for examining whether eye-movement patterns during a visual search task can be modified in response to hidden contingencies. Because of the phenomenon of change blindness, it is possible to surreptitiously introduce a target into a search display mid-trial, during a saccadic eye movement, so that it appears directly in the line of gaze at the termination of the saccade. From the participant's point of view, the target is simply found on the screen. From the experimenter's point of view, the introduction of the target can be made contingent on particular patterns of eye-movement. In a series of preliminary studies, I found that I could make participants produce larger saccades sooner, if I surreptitiously made target appearance in a visual search task contingent (probabilistically) on large eye movements. That is, in time-limited search trials, success rate at "finding" (eliciting) the target was found to increase over the first 60 trials, and this was related primarily to a decreasing latency for making a large eye-movement during a trial. Eye-movement patterns for controls did not change with time.
The goal of the present investigations was to look at somewhat more complex rules for target elicitation. Specifically, in each of the two experiments to be presented here, an attempt is made to promote either clockwise or counter-clockwise search patterns in a dense display. In Experiment 1, the rules for the clockwise and counterclockwise groups were defined with respect to successive saccade directions. If the change in direction between two saccades made a turn to the right, the clockwise rule was satisfied; if it made a turn to the left, it counted as a counterclockwise. In Experiment 2, the rules for the clockwise and counterclockwise search groups were defined with respect to the screen (i.e., on the right side of the screen a downward saccade would be clockwise, whereas on the left of the screen, an upward saccade would be clockwise). These rules are schematically illustrated in figure 16.1. Intriguingly, implicit learning will be demonstrated in Experiment 1, but not in Experiment 2. Conversely, in Experiment 2 several participants become explicitly aware of a successful search strategy for finding the target, whereas none did in Experiment 1. These findings suggest that implicit learning in eye-movement control systems may be limited to variables associated with the coding of saccades in eye-centered polar coordinates, rather than in world (or display) coordinates, whereas conscious strategies are best developed for world- or display-relative coordinate systems.
[FIGURE 16.1 HERE; figures not yet available]
Apparatus. The displays were controlled by a Macintosh PowerPC 7600 and presented on a ViewSonic 17 RGB monitor with a resolution of 1152 x 870 pixels. Vertical refresh was 75 Hz. The display was viewed from a distance of about 0.5 m without head restraint. An SRR EyelinkÆ which uses head-mounted 250 Hz infrared video and head movement compensation to sample gaze position at 4 ms intervals monitored eye position. Physical updating of the display could be accomplished within a single video frame (13.3 ms), for a total lag of less than 18 ms. Gaze accuracy was normally well within 0.5 degrees, and the display-center gaze position was recalibrated at the beginning of each trial to avoid systematic drift.
Design. Each participant performed visual search in 150 trials. Participants were randomly assigned to either the clockwise rule condition (N = 9) or the counterclockwise rule condition (N = 11). Displays. Search displays were composed of 800 randomly scattered, nonoverlapping line segments (approx. 2 x 24 min of arc), appearing, with equal frequency, in red, green, blue, yellow and in each of 4 orientations (0, 45, 90 135 deg) against a black background. The target was a red "X" composed of two diagonal lines, (which was not initially present on experimental trials).
Rule for target elicitation. The implicit rules for target elicitation were intended to foster either clockwise or counterclockwise search patterns around the screen by rewarding pairs of saccades that constituted either a right-hand turn or a left-hand turn, respectively. In essence, any large saccade (reaching a velocity of at least 300 deg/sec over an 8 ms period) was treated as the first leg of a turn and immediately subsequent saccades were compared in direction with the first leg. Any turn between 45 and 135 deg was considered clockwise, and turns of 225 to 315 deg were considered counterclockwise. Target elicitation was thereafter guaranteed provided that a candidate target location was available within a deg of final fixation. Since saccadic movements can be detected and their direction well characterized by triplets of successive gaze samples (4 ms apart) which show large absolute changes in position, satisfaction of the direction-change rule could be computed during the second saccade. A target could then be placed near the anticipated landing point of the saccade provided two further conditions were met. Namely, a target could only appear in a location previously occupied by one of the red elements on the screen, and, to avoid detection of the deception, targets could not appear in a location within 2 degrees of any previous fixation position. If no appropriate location was available, target elicitation was delayed until some further set of saccades satisfied the rule.
The ostensible task. Participants were told that they were in a study of eye-movements during visual search. This served as a cover story for the use of the head-mounted eye tracker. Their task was to find a red "X" on the screen if one was present and to press a button as soon as they found it. They began each trial fixating a spot in the center of the screen and pressing a key which triggered the start of the trial 500 msec later. Trials were always terminated when the response button was pressed or, if no button was pressed, after 3 seconds.
Assessment of awareness. All participants were interviewed at the conclusion of the experiment. Several believed that the target was not always present from the beginning, but none believed that target appearance was in any way connected with their search strategy or eye-movements. Only one student mentioned a correlative strategy of examining the corners of the display first (though he did not indicate that he had swept the corners in any particular direction). The results discussed below are unchanged when this student's data are dropped from the analysis.
Because learning curves are often decelerating functions, the analysis of learning was conducted over geometrically increasing numbers of trials. Specifically, the 150 trials were broken into an initial block of ten, subsequent blocks of twenty and forty trials, and a final block of eighty trials. (Analyses by blocks composed of equal number of trials came to equivalent conclusions.) The dependent measure used to assess improvement over time is simply the rate of search success (number of successful searches divided by the number of trials in each block).
To establish that the learning was specific to the hidden contingencies, a second analysis was conducted to determine whether the two experimental groups differed in their eye-movement patterns. Because trials were terminated upon target discovery, it was necessary to perform the statistical tests of saccade-direction frequency only on the initial portions of trials. A cut-off of 800 msec was chosen, because very few responses were ever generated before this time had elapsed. Only saccades completed prior to this time during each trial were considered. The measure used to assess differential learning of the directional rules was the frequency of clockwise and of counterclockwise saccades per trial. Because the original intent of the experiment was to foster screen-relative search patterns, these directions were defined with respect to the display itself, for purposes of analysis, rather than in the terms used trigger the targets. All saccades with a peak velocity of at least 300 deg/s were checked. If their direction at their midpoint was within 45 degrees of being perpendicular to a line from the center of the display, then they were categorized as either clockwise or counterclockwise.
[FIGURE 16.2 HERE; figures not yet available]
The frequency of such saccades during the final block of trials was subjected to a repeated measures ANOVA with Saccade Direction as a within-group factor and Rule Direction as a between-group factor. Differential learning would be indicated by an interaction between Rule Direction and Saccade Direction. In fact, as illustrated in figure 16.2b, this interaction was reliable F(1,18) = 6.28, p < .05. Overall, the frequency of rule-consistent saccades was 0.715 per trial, whereas the frequency of oppositely directed saccades (i.e., rule irrelevant) was only 0.518 per trial.
In conclusion, the results of this experiment demonstrate clear evidence of learned sensitivity to specific hidden contingencies. Our interviews with participants indicated that none of them imagined that target appearance was in any way caused by their actions. Their eye-movement patterns nonetheless indicate a learned sensitivity to the eye-movement-contingency embedded in the experimental task.
Assessment of awareness: In addition to questions concerning strategies used, all participants were asked to guess what the underlying rule was after we revealed that a rule had been in operation. They were then told that the rule hinged on either clockwise or counterclockwise eye motions and asked to indicate which direction they had been trained with. Three participants in the clockwise search conditions described strategies of sweeping around the screen prior to being informed of the rule. All three had been in the clockwise rule condition and correctly indicated this. Data from these students will be left out of the main analysis (Two of them were the two most successful at the task overall.) When asked to guess what the rule might have been four more of the participants came up with hypotheses that were correlated with the actual search rule. Three of these four students chose the correct rule direction. Of the remaining 15 participants, only 6 chose the correct direction.
To confirm that the results of Experiment 2 differed from those of Experiment 1, data from both experiments were analyzed together in a repeated measures ANOVA with Rule Type (Eye-centric or Display-based) and Rule Direction as between-group variables and Trial Block as a within-group variable. As expected, the effect of trial Block differed reliably as a function of Rule Type, F(3,108) = 3.41, p < .05. The simplest interpretation of these results is that implicit learning did not occur in Experiment 2, whereas it clearly did in Experiment 1.
On the other hand, seven of the 22 students in this experiment were able to articulate explicit strategies that were correlated with the actual rule, compared with only one out of 20 in Experiment 1, c2 = 7.68, p < .01. Apparently, the rule was not intrinsically more difficult to learn, though it evidently was not learned by any implicit mechanisms. Indeed, it is a more easily articulated rule, readily available to explicit awareness.
Previous examples of dissociations between implicit learning and explicit awareness include Berry and Broadbent's (1984) classic sugar production experiment. In their study with a hidden rule, explicit instruction failed to improve task performance, though implicit understanding developed in the uninstructed. In that experiment, the task is presented as a problem to be solved. An important difference between the sugar production task and this one is that participants in the visual search task are unaware that there is even a rule to be learned. From their conscious perspective the ostensible task is transparent in Experiment 1.
Although the training rules differed for the two experiments, the same display-relative rule was used to interpret both sets of eye-movement data. It seems therefore all the more surprising that the very same dependent measure of the rule can turn up such different results. If the first rule can be implicitly learned, and the second rule would have been satisfied more frequently by satisfaction of the first rule, why didn't the participants in Experiment 2 simply develop the same implicit strategy as those in Experiment 1? It is possible that intervening rule steps needed to be learned, but it is also possible that the consciously available structure of the task (absence of targets in the middle) somehow interfered with the implicit learning process.
Several participants in Experiment 2 commented that the target never appeared in the middle of the display (a consequence of the display-based rule definition). Perhaps the rotary search strategy they consciously adopted (or which occurred to them even if they did not implement it successfully) was a response to this salient feature of the search environment.
These experiments were intended to study implicit learning in eye-movement control systems that might facilitate the acquisition of visual information. Although the conclusions reached here are tentative, rules based on eye-centered coordinate frames were more susceptible to implicit learning than were display-centered rules. Further research is needed to determine whether this finding is an artifactual result of salient differences between the search tasks, or whether it indeed signals an important limitation on implicit learning in eye-movement control systems.
Berry, D. and Z. Dienes. 1993. Implicit learning: theoretical and empirical issues. Hillsdale, N.J.: L. Erlbaum Associates.
Durgin, F. H., and D. R. Proffitt. 1996. Visual learning in the perception of texture: simple and contingent aftereffects of texture density. In Spatial Vision 9:423-474.
Gibson, J. J. 1966. The senses considered as perceptual systems. Boston, Houghton Mifflin.
Grimes, J. 1996. On the failure to detect changes in scenes across saccades. In K. A. Akins (Ed), Perception (pp. 89-110). New York: Oxford University Press.
Reber, A. S. 1993. Implicit learning and tacit knowledge: an essay on the cognitive unconscious. New York: Oxford University Press.
Rensink, R. A., J. K. O'Regan, and J. J. Clark. 1997. To see or not to see: The need for attention to perceive changes in scenes. Psychological Science 8:368-373.