Moderately complex features

There is a principal difficulty in determining the stimulus selectivity of individual cells in TE. There is a great variety of object features in the world, and it remains to be determined how the brain scales down this variety. There have been studies that used mathematically perfect sets of shapes (Gallant et al., 1993, 1996; Richmond et al., 1987; Schwartz et al., 1983). However, the generality of these sets would hold only if the system were linear, which is hardly expected in higher visual centers.

We have used an empirical reduction method that involves the real-time modification of stimulus images on an image-processing computer system (Fujita et al., 1992; Ito et al., 1994, 1995; Kobatake and Tanaka, 1994; Tanaka et al., 1991; Wang et al., 1998). After spike activities from a single cell were isolated, many three-dimensional animal and plant models were first presented manually to find the effective stimuli. Different aspects of the objects were presented in different orientations. Second, images of several most effective stimuli were taken with a video camera and displayed on a TV monitor by a computer to determine the stimulus that evoked the maximal response. Third, the image of the most effective stimulus was simplified step by step in the direction in which the maximal activation was maintained. Finally, the minimal requirement for maximal activation was determined as the critical feature for the cell, as exemplified in Figure 77.2. Even starting at the same object image, the effective direction of simplification varied from cell to cell. Thus, images used in the simplification procedure were made in real time while the activity of the cell was recorded. The procedure is time-consuming, and it usually takes 2 to 4 hours to determine the critical feature for one TE cell. The magnitude of responses often increased as the complexity of an image was reduced. This may be due to the adjustment of size, orientation, and shape, as well as the removal of other features, which may suppress the activation by the critical feature (Missal et al., 1997; Sato, 1989, 1995; Tsunoda et al., 2001).

Figure 77.2..  

Example of reductive determination of optimal features for a cell recorded in TE.

Additional examples of the reduction of complexity of images for 12 other TE cells are shown in Figure 77.3. The pictures to the left of the arrows are the original images of the most effective object stimuli, and those to the right are the critical features determined after the reduction process. Some of the critical features are moderately complex shapes, while others are combinations of such shapes with color or texture. After determining the critical features for hundreds of cells in TE, we concluded that most cells in TE required moderately complex features for their maximal activation. The critical features for TE cells were more complex than just the orientation, size, color, or simple textures, which are known to be extracted and represented by cells in V1, but at the same time were not sufficiently complex to represent the image of a natural object through the activity of single cells. The combined activation of multiple cells, which represent different features contained in the object image, is necessary.

Figure 77.3..  

Further examples of reductive determination of optimal features for 12 other TE cells. The images to the left of the arrows represent the original images of the most effective object stimulus, and those to the right of the arrows represent the critical features determined by the reduction process. (See color plate 54).

Although the reduction method appears to be the best among currently available methods of determining the stimulus selectivity of TE cells, it has limitations. The initial survey of effective stimuli cannot cover the entire variety of objects existing in the world. We may miss some very effective features. In addition, the tested methods of reducing the complexity of effective object images are limited by the time of continuous recording from a single cell and also by the imagination of the experimenter. Because of these limitations, the objectiveness of the determined optimal features has sometimes been doubted. It is desirable that the reduction procedure be automated. Pasupathy and Connor (2001) have developed a method of presenting a large number of shapes made by combining several arcs of different curvatures. They have shown the usefulness of this method in studying the selectivity of V4 cells, but it may not be useful for TE cells, which respond to more complicated shapes than do V4 cells. Keysers et al. (2001) developed a method of analyzing responses to more than 1000 stimulus images in a fixation task. The stimulus images were presented individually for a short time (e.g., 100 msec) without an interstimulus interval. The following stimulus presented may inhibit the response to the previous stimulus, but because the order of stimulus presentation is randomized and because TE cells tend to respond to a small part of the stimuli, there are no inhibitory interactions in the majority of repetitions. These two methods may be combined to explore systematically a large feature space of complex shapes—sufficiently complex for the activation of most TE cells.