| |
Abstract:
We compare discriminative and generative learning as typified
by logistic regression and naive Bayes. We show, contrary to a
widely held belief that discriminative classifiers are almost
always to be preferred, that there can often be two distinct
regimes of performance as the training set size is increased, one
in which each algorithm does better. This stems from the
observation -- which is borne out in repeated experiments -- that
while discriminative learning has lower asymptotic error, a
generative classifier may also approach its (higher) asymptotic
error much faster.
|
