This is the fourth edition of The Cognitive Neurosciences. With each succeeding edition comes the same question: Why does a volume on cognition include sensation and perception—and a double dose at that? The answers to these questions lie in the chapters of this section.

For more than a century, there has been a systematic quest to understand what and how physical information is captured by the nervous system. A great gift of 20th-century neuroscience is a set of beautiful and definitive experiments that explain much about this peripheral encoding. Through the use of illusions and demonstrations, introductory textbooks rightly emphasize that these peripheral signals must be interpreted for their biological significance. For example, the signals encoded in the retina do not measure the distance to a predator; the signals encoded in the auditory nerve do not specify the significance of a sound. Rather, as Helmholtz taught us, the brain uses the receptor signals to make inferences about information. The ongoing work in sensation and perception has shifted to focus on questions about the brain's computational circuitry: What are the computations performed on raw sense data by the sensory systems? What quantities of interest are extracted by these computations? What neural circuits perform them, and how do these circuits work?

Science is frequently imprisoned by technology, and neuroscience is no exception. Our view of brain function has been conditioned and distorted by the methods that we have available. In sensory neuroscience, the dominant technique for most of the last quarter-century has been single-unit recording. Unit recording experiments offer unparalleled access to the details of neural computation and allow us to measure activity at a fundamental computational scale of the system. But the resolution of unit recording comes at a cost. Brain function depends on the organization and transmission of information over distances far larger than a single electrode can sense, and we have had a difficult time obtaining a view of the nervous system's actions at long spatial scales. Our view is expanding on the basis of new methods for measuring neural signals at multiple length scales. There are noninvasive methods to measure the spatial organization of cone photoreceptors in the living human eye and new methods to measure spatially resolved activity in the human brain. Techniques for studying cells now measure dozens of cells at once and can even track important activation-related signals from local clusters of thousands of neurons. These advances all provide valuable information about the way in which sensory signals are encoded by populations of neurons and how these signals are interpreted within the brain. The new measurements are accompanied by developments in computational principles and tools that help us to understand how the nervous system extracts biologically relevant events from a complex array of environmental signals. The chapters in this section show how the combination of new measurements and analytic methods are being combined to create a deeper understanding of how the nervous system represents biologically relevant sensory events.

The topics in this section span vision, audition, olfaction, analysis, and theory. The reader will find a wide array of techniques, including anatomy, electrophysiology, optical imaging, neuroimaging, behavior, and computation. The close connection between computation and methods is nicely illustrated in the chapters concerned with peripheral encoding. Brainard considers how one might make unexpected inferences about object color from the encoding by the three types of cones; Carroll, Yoon, and Williams analyze the information contained in the cone spatial mosaic. The computational analysis of neural encoding is fundamental across sensation. Yeshurun and colleagues describe how psychophysical and computation methods are used to understand the organization of olfaction. Richards and Kidd introduce the reader to questions about how the auditory system disentangles sounds in a complex environment. Two theoretical chapters, by Simoncelli and Barlow, consider approaches to analyzing information in sensory arrays. Simoncelli offers a modern approach that builds on the fundamental work of important pioneers; Barlow, one of these pioneers, gives us a historical perspective and puts his current thinking in that context.

Much important work remains to be done at the single-neuron level, and each of the chapters by Connor and colleagues, DeAngelis, and Angelaki and colleagues offers a snapshot of the state of the art. Connor and colleagues show how the rigorous application of quantitative analysis methods can begin to offer an account of the transformation of information about visual sensory elements in V1 into a more subtle and sophisticated representation of object features in temporal cortex. DeAngelis considers area MT, best known for its contributions to visual motion perception, and gives an account that combines unit recording and behavioral analysis to demonstrate that MT also plays a key role in the perception of depth. Angelaki and colleagues remind us that visual signals are processed in combination with information from other senses, and describes a series of experiments showing the coordinated role that vestibular and visual signals play in representing information about motion through the world in areas that used to be considered “purely” visual. There are intimate links between sensory and motor systems, none closer than the link between the visual system and the oculomotor system. Morrone and Burr review behavioral and functional imaging work, addressing the ways in which these systems work together to compute our stable perceptual experience from a highly unstable retinal signal.

Many researchers now seek to take data from the unit level and generalize it into a form that can be used to account directly for behavior. Ringach reexamines the specificity of neuronal response in the most-studied of all sensory areas—V1—and shows unexpected relationships between that specificity and the cortex's functional architecture. Seidemann and colleagues also start with the familiar visual representation in V1 but use optical imaging techniques to ask about population activity on a scale of millimeters rather than microns. Drawing on his recordings, on simultaneous behavioral measurement, on knowledge from unit work, and on the theory of encoding and representation, he constructs an account of simple visual performance.

The revolution in functional MRI, enabling scientists to make spatially resolved measurements of the awake-behaving human brain, has also played an important role in sensation and perception. The chapters by McKone, Griffiths, and Goebel and their colleagues describe approaches to interpreting signals in human visual cortex. Taking a developmental approach, McKone and colleagues seek to understand how the pathways that are needed to recognize and interpret faces develop. Griffiths and colleagues apply computational methods to understanding the structure of sounds and then further considers ideas about how the responses spread across cortex might interpret these sounds. Goebel and De Weerd examine the process of image interpolation (filling-in) in visual cortex. Notions of interpretation are often connected with neural plasticity (see Section II), which also plays a major role in sensory processing. Wright and Zhang document a series of behavioral studies showing how auditory information processing depends on training and experience, suggesting that much of the cortical machinery studied by others can adapt its function to suit the needs of the organism.

Taken together, these contributions show that sensation and perception are intertwined with cognition in two fundamental ways. First, the work in sensation and perception crosses borders between many fields, integrating work in behavior, neuroscience, and computation. This field offers an excellent testing ground to evaluate many of the techniques that will be needed as we develop the field of cognitive neuroscience. Second, the chapters in this section show clearly that the information that sensory and perceptual pathways provide to cognition sets critical bounds on the information that is available to the brain. These are limits that influence the thoughts we have, the decisions we make, and the emotions we experience. Perception provides three dimensions of color and three dimensions of space; there are molecules whose smell evokes pleasure or disgust. Sensation and perception serve the needs of cognition; cognition, in turn, works within the bounds set by perception.