Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

July 2011, Vol. 23, No. 7, Pages 1899-1909
(doi: 10.1162/NECO_a_00137)
© 2011 Massachusetts Institute of Technology
Determination and the No-Free-Lunch Paradox
Article PDF (177.13 KB)
Abstract

We discuss the no-free-lunch NFL theorem for supervised learning as a logical paradox—that is, as a counterintuitive result that is correctly proven from apparently incontestable assumptions. We show that the uniform prior that is used in the proof of the theorem has a number of unpalatable consequences besides the NFL theorem, and propose a simple definition of determination (by a learning set of given size) that casts additional suspicion on the utility of this assumption for the prior. Whereas others have suggested that the assumptions of the NFL theorem are not practically realistic, we show these assumptions to be at odds with supervised learning in principle. This analysis suggests a route toward the establishment of a more realistic prior probability for use in the extended Bayesian framework.