An Introduction to Neural Networks


An Introduction to Neural Networks falls into a new ecological niche for texts. Based on notes that have been class-tested for more than a decade, it is aimed at cognitive science and neuroscience students who need to understand brain function in terms of computational modeling, and at engineers who want to go beyond formal algorithms to applications and computing strategies. It is the only current text to approach networks from a broad neuroscience and cognitive science perspective, with an emphasis on the biology and psychology behind the assumptions of the models, as well as on what the models might be used for. It describes the mathematical and computational tools needed and provides an account of the author's own ideas.

Students learn how to teach arithmetic to a neural network and get a short course on linear associative memory and adaptive maps. They are introduced to the author's brain-state-in-a-box (BSB) model and are provided with some of the neurobiological background necessary for a firm grasp of the general subject.

The field now known as neural networks has split in recent years into two major groups, mirrored in the texts that are currently available: the engineers who are primarily interested in practical applications of the new adaptive, parallel computing technology, and the cognitive scientists and neuroscientists who are interested in scientific applications. As the gap between these two groups widens, Anderson notes that the academics have tended to drift off into irrelevant, often excessively abstract research while the engineers have lost contact with the source of ideas in the field. Neuroscience, he points out, provides a rich and valuable source of ideas about data representation and setting up the data representation is the major part of neural network programming. Both cognitive science and neuroscience give insights into how this can be done effectively: cognitive science suggests what to compute and neuroscience suggests how to compute it.

Table of Contents

  1. Introduction
  2. Acknowledgements
  3. 1. Properties of Single Neurons
  4. 2. Synaptic Integration and Neuron Models
  5. 3. Essential Vector Operations
  6. 4. Lateral Inhibition and Sensory Processing
  7. 5. Simple Matrix Operations
  8. 6. The Linear Associator: Background and Foundations
  9. 7. The Kinear Associator: Simulations
  10. 8. Early Network Models: The Perceptron
  11. 9. Gradient Descent Algorithms
  12. 10. Representation of Information
  13. 11. Applications of Simple Associators: Concept Formation and Object Motion
  14. 12. Energy and Neural Networks: Hopfield Networks and Boltzmann Machines
  15. 13. Nearest Neighbor Models
  16. 14. Adaptive Maps
  17. 15. The BSB Model: A Simple Nonlinear Autoassociative Neural Network
  18. 16. Associative Computation
  19. 17. Teaching Arithmetic to a Neural Network
  20. Afterword
  21. Index