Monthly
288 pp. per issue
6 x 9, illustrated
ISSN
0899-7667
E-ISSN
1530-888X
2014 Impact factor:
2.21

Neural Computation

November 15, 1999, Vol. 11, No. 8, Pages 1885-1892
(doi: 10.1162/089976699300016007)
© 1999 Massachusetts Institute of Technology
Combined 5 × 2 cv F Test for Comparing Supervised Classification Learning Algorithms
Article PDF (126.81 KB)
Abstract

Dietterich (1998) reviews five statistical tests and proposes the 5 × 2 cv t test for determining whether there is a significant difference between the error rates of two classifiers. In our experiments, we noticed that the 5 × 2 cv t test result may vary depending on factors that should not affect the test, and we propose a variant, the combined 5 × 2 cv F test, that combines multiple statistics to get a more robust test. Simulation results show that this combined version of the test has lower type I error and higher power than 5 × 2 cv proper.