Framework

The receptive field is a central construct in virtually all physiological and computational accounts of the workings of the early visual system. It describes how visual inputs are transformed into neural responses and thus represents the computations performed by a neuron. For this reason, studies of receptive fields are essential for understanding the neural basis of visual perception.

The receptive field of a visual neuron was originally defined as the region of the retina that must be illuminated in order to evoke a response from the neuron (Hartline, 1940). In addition to defining the spatial extent of the receptive field, traditional descriptions specified that particular regions of a receptive field could respond to luminance increments (ON subregions), luminance decrements (OFF subregions), or both (ON-OFF subregions). Using this basic scheme, the spatial organization of receptive fields in the retina, lateral geniculate nucleus (LGN), and striate cortex was elucidated in a series of classic studies (e.g., Barlow, 1953; Hubel and Wiesel, 1961, 1962; Kuffler, 1953), some of which are described below. However, research over the past two decades has revealed a number of response properties that are not easily described within the basic scheme of ON and OFF subregions, suggesting that an expanded view of the receptive field is needed.

There are two major limitations to the traditional characterization of receptive fields. One is that it does not describe the temporal dynamics of neuronal responses. The response of a neuron to a very brief stimulus rises after some delay and then decays over time, and the time course of this response varies among neurons in the early visual pathways (e.g., DeAngelis et al., 1993a; De Valois et al., 2000; Wolfe and Palmer, 1998). In addition, the temporal dynamics may vary dramatically from one spatial location to another within the classically defined receptive field (DeAngelis et al., 1993a; McLean et al., 1994). Thus, it is essential for any modern description of receptive fields to incorporate both spatial and temporal dimensions.

The second major limitation of the traditional description of receptive fields is that it does not allow for a rigorous characterization of nonlinear response properties. The response of a neuron to multiple stimuli distributed across space and time is often not equal to the linear summation of the responses to the individual stimuli, and in many cases the dominant component of the response is due to nonlinear interactions among stimuli. Thus, a modern description of the receptive field must be able to represent nonlinear response properties in addition to linear characteristics.

In this chapter, we define a receptive field as the spatiotemporal structure that characterizes the transformation between the visual image and a neuron's response, a view that has emerged from research done over the past few decades in numerous laboratories. The main advantage of this modern view is that it describes temporal dynamics of neuronal responses, and it considers the information processing performed by neurons through spatiotemporal interactions (both linear and nonlinear) within the classical receptive field. We show that this view provides new and better explanations for some of the response properties of neurons in primary visual cortex. It should be noted, however, that we do not deal with response modulations originating from outside the classical receptive field, which are described elsewhere in this volume (Chapter 45). Although nonclassical surround effects constitute important receptive field properties, they are not generally elicited by the types of stimuli discussed here and will not be a part of our descriptions.

To characterize the transformation from visual image to neuronal response, it is useful to start by considering the relevant dimensions of the visual input (Adelson and Bergen, 1991). Ignoring color for the sake of simplicity, the image formed on the retina of each eye is a time (T)-varying pattern of luminance across two spatial dimensions (X,Y), which we denote as I(X,Y,T). The response (instantaneous spike rate) of a neuron, on the other hand, is a function of only time and will be represented here by R(T). In our framework, the receptive field transforms the image, I(X,Y,T), into the neuronal response, R(T), in two possible ways: linearly or nonlinearly (Fig. 44.1A, B). In the linear case, the neuron's response, R(T), is described by a convolution of the image, I(X,Y,T), with the spatiotemporal receptive field, denoted by RF_L(X,Y,T):

Figure 44.1..  

A visual neuron can be viewed as a system that takes an image, I(X,Y,T), as input and produces a response, R(T), as output. I(X,Y,T) represents a luminance distribution over two dimensions of space (X,Y) and time (T), whereas R(T) denotes the instantaneous spike rate as a function of time. The receptive field of a visual neuron can then be defined as a spatiotemporal structure that characterizes the transformation between the image, I(X,Y,T), and the response, R(T). A, For a linear neuron, this transformation is a linear summation of the image I(X,Y,T) weighted by RF_L(X,Y,T), which we will call a linear receptive field map. B, For a nonlinear neuron, the response is the sum of a linear component due to RF_L(X,Y,T), and a nonlinear component due to interactions among visual inputs over space and time. Transformations between combinations of inputs and nonlinear responses are represented by nonlinear receptive field maps, RF_Np, where the subscript p indicates the order of nonlinear interaction. For example, the second-order nonlinear interaction between two inputs, i and j, is denoted by RF_N2(X_i,Y_i,T_i, X_j,Y_j,T_j). C, Space-time separable and inseparable receptive fields. Each contour plot shows a spatiotemporal receptive field map, RF_L(X,T), for a hypothetical V1 simple cell (the Y-axis of RF_L(X,Y,T) has been eliminated for simplicity). Solid and dashed contours represent responses to luminance increments (bright stimuli) and decrements (dark stimuli), respectively. The receptive field map on the left is space-time separable, meaning that RF_L(X,T) can be described as a product of a spatial function G(X) and a temporal function H(T), that is, RF_L(X,T) = G(X)H(T). The map on the right is inseparable; it cannot be decomposed into a spatial function and a temporal function.