Online fitting of computational cost to environmental complexity: Predictive coding with the ε-network

Conference Date
Lyon, France
Date Published
September 2017
Conference Date: 2017, Vol. 14, Pages 380-387.
(doi: 10.7551/ecal_a_065)
© 2017 Massachusetts Institute of Technology Published under a Creative Commons Attribution 4.0 International (CC BY 4.0) license
Article PDF (2.1 MB)

We propose the Epsilon Network (ε-network), a network that automatically adjusts its size to the complexity of a stream of data while performing online learning. The network optimises its topology during training, simultaneously adding and removing neurons and weights: it adds neurons where they can raise performance, and removes redundant neurons while preserving performance. The network is a neural realisation of the ε-machine devised by Crutchfield and al. (Crutchfield and Young (1989)). In this paper our network is trained to predict video frames; we evaluate it on simple, complex, and noisy videos and show that the final number of neurons is a good indicator of the complexity and predictability of the data stream.