Foundations of Research
Volume 1

Researchers will find Neurocomputing an essential guide to the concepts employed in this field that have been taken from disciplines as varied as neuroscience, psychology, cognitive science, engineering, and physics. A number of these important historical papers contain ideas that have not yet been fully exploited, while the more recent articles define the current direction of neurocomputing and point to future research. Each article has an introduction that places it in historical and intellectual perspective.

Included among the 43 articles are the pioneering contributions of McCulloch and Pitts, Hebb, and Lashley; innovative work by Von Neumann, Minsky and Papert, Cooper, Grossberg, and Kohonen; exciting new developments in parallel distributed processing.

Table of Contents

  1. General Introduction
  2. 1. (1890) William James, Psychology (Briefer Course), New York: Holt, Chapter XVI, "Association," pp. 253-279.
  3. 2. (1943) Warren S. McCulloch and Walter Pitts, "A logical calculus of the ideas immanent in nervous activity," Bulletin of Mathematical Biophysics 5:115-133.
  4. 3. (1947) Walter Pitts and Warren S. McCulloch, "How we know universals: the perception of auditory and visual forms," Bulletin of Mathematical Biophysics 9:127-147.
  5. 4. (1949) Donald O.Hebb, The Organization of Behavior, New York: Wiley, Introduction and Chapter 4, "The first stage of perception: growth of the assembly," pp. xi-xix, 60-78.
  6. 5. (1950) K. S. Lashley, "In search of the engram," Society of Experimental Biology Symposium, No. 4: Psychological Mechanisms in Animal Behavior, Cambridge: Cambridge University Press, pp. 454-455, 468-473, 477-480.
  7. 6. (1956) N. Rochester, J. H. Holland, L. H. Habit, and W. L. Duda, "Tests on a cell assembly theory of the action of the brain, using a large digital computer," IRE Transactions on Information Theory IT-2: 80-93.
  8. 7. (1958) John von Neumann, The Computer and the Brain, New Haven: Yale University Press, pp. 66-82.
  9. 8. (1958) F. Rosenblatt, "The perceptron: a probabilistic model for information storage and organization in the brain," Psychological Review, 65:386-408.
  10. 9. (1958) O. G. Selfridge, "Pandemonium: a paradigm for learning," Mechanisation of Thought Processes: Proceedings of a Symposium Held at the National Physical Laboratory, November 1958, London: HMSO, pp. 513-526.
  11. 10. (1960) Bernard Widrow and Marcian E. Hoff, "Adaptive switching circuits," 1960 IRE WESCON Convention Record, New York: IRE, pp. 96-104.
  12. 11. (1962) H. D. Block, "The Perceptron: a model for brain functioning. I," Reviews of Modern Physics 34:123-135.
  13. 12. (1969) D. J. Willshaw, O. P. Buneman, and H. C. Longuet-Higgins, "Non-holographic associative memory," Nature 222:960-962.
  14. 13. (1969) Marvin Minsky and Seymour Papert, Perceptrons, Cambridge, MA: MIT Press, Introduction, pp. 1-20, and p. 73 (figure 5.1).
  15. 14, 15. (1972) Teuvo Kohonen, "Correlation matrix memories," IEEE Transactions on Computers C-21: 353-359.(1972) James A. Anderson, "A simple neural network generating an interactive memory," Mathematical Biosciences 14:197-220
  16. 16. (1973) L. N. Cooper, "A possible organization of animal memory and learning," Proceedings of the Nobel Symposium on Collective Properties of Physical Systems, B. Lundquist and S. Lundquist (Eds.), New York: Academic Press, pp. 252-264.
  17. 17. (1973) Chr. von der Malsburg, "Self-organization of orientation sensitive cells in the striate cortex," Kybernetik 14:85-100.
  18. 18. (1975) W. A. Little and Gordon L. Shaw, "A statistical theory of short and long term memory," Behavioural Biology 14:115-133.
  19. 19. (1976) S. Grossberg, "Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors," Biological Cybernetics 23:121-134.
  20. 20. (1976) D. Marr and T. Poggio, "Cooperative computation of stereo disparity," Science 194:283-287.
  21. 21. (1977) S.-I. Amari, "Neural theory of association and concept-formation," Biological Cybernetics 26:175-185.
  22. 22. (1977) James A. Anderson, Jack W. Silverstein, Stephen A. Ritz, and RAndall S. Jones, "Distinctive features, categorical perceptron, and probability learning: some applications of a neural model," Psychological Review 84:413-451.
  23. 23. (1978) Scott E. Brodie, Bruce W. Knight, and Floyd Ratliff, "The response of the Limulus retina to moving stimuli: a prediction by Fourier synthesis," Journal of General Psychology 72:129-154, 162-166.
  24. 24. (1980) Stephen Grossberg, "How does a brain build a cognitive code?" Psychological Review 87:1-51.
  25. 25. (1981) James L. McClelland and David E. Rumelhart, "An interactive activation model of context effects in letter perception: part 1. An account of basic findings," Psychological Review, 88:375-407.
  26. 26. (1982) Elie L. Bienenstock, Leon N. Cooper, and Paul W. Munro, "Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex," Journal of Neuroscience 2:32-48.
  27. 27. (1982) J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy of Sciences 79:2554-2558.
  28. 28. (1982) David Marr, Vision, San Francisco: W. H. Freeman, pp. 19-38, 54-61.
  29. 29. (1982) J. A. Feldman and D. H. Ballard, "Connectionist models and their properties," Cognitive Science 6:205-254.
  30. 30. (1982) Teuvo Kohonen, "Self-organized formation of topologically correct feature maps," Biological Cybernetics 43:59-69.
  31. 31. (1983) Kunihiko Fukushima, Sei Miyake, and Takayuki Ito, "Neocognitron: a neural network model for a mechanism of visual pattern recognition," IEEE Transactions on Systems, Man, and Cybernetics SMC-13:826-834.
  32. 32. (1983) Andrew G. Barto, Richard S. Sutton, and Charles W. Anderson, "Neuronlike adaptive elements that can solve difficult learning control problems," IEEE Transactions on Systems, Man, and Cybernetics SMC-13:834-846.
  33. 33. (1983) S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi, "Optimization of simulated annealing," Science 220:671-680.
  34. 34. (1984) Francis Click, "Function of the thalamic reticular complex: the searchlight hypothesis," Proceedings of the National Academy of Sciences 81:4586-4590.
  35. 35. (1984) J. J. Hopfield, "Neurons with graded response have collective computational properties like those of two-state neurons," Proceedings of the National Academy of Sciences 81:3088-3092.
  36. 36. (1984) Andrew G. Knapp and James A. Anderson, "Theory of categorization based on distributed memory storage," Journal of Experimental Psychology: Learning, Memory, and Cognition 10:616-637.
  37. 37. (1984) Stuart Geman and Donald Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-6:721-741.
  38. 38. (1985) David H. Ackley, Geoffrey E. Hinton, and Terrence J. Sejnowski, "A learning algorithm for Boltzmann machines," Cognitive Science 9:147-169.
  39. 39. (1985) Nabil H. Farhat, Demetri Psalitis, Aluizio Prata, and Eung Pake, "Optical implementation of the Hopfield model," Applied Optics 24:1469-1475.
  40. 40. (1986) Terrence J. Sejnowski and Charles R. Rosenberg, "NETtalk: a parallel network that learns to read aloud," The Johns Hopkins University Electrical Engineering and Computer Science Technical Report JHU/EECS-86/01, 32 pp.
  41. 41, 42. (1986) D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propogation," Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Vol. I, D. E. Rumelhart and J. L. McClelland (Eds.) Cambridge, MA: MIT Press, pp. 318-362.(1986) David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams, "Learning representations by back-propogating errors," Nature 323:533-536.
  42. 43. (1987) Massimo A. Sivilotti, Michelle A. Mahowald, and Carver A. Mead, "Real-time visual computations using analog CMOS proceeding arrays," Advanced Research in VLSI: Proceedings of the 1987 Stanford Conference, P. Losleben (Ed.), Cambridge, MA: MIT Press, pp. 295-312.
  43. Afterword
  44. Name Index
  45. Subject Index