| |
Abstract:
AdaBoost and other ensemble methods have successfully been
applied to a number of classification tasks, seemingly defying
problems of overfitting. AdaBoost performs gradient descent in an
error function with respect to the margin, asymptotically
concentrating on the patterns which are hardest to learn. For very
noisy problems, however, this can be disadvantageous. Indeed,
theoretical analysis has shown that the margin distribution, as
opposed to just the minimal margin, plays a crucial role in
understanding this phenomenon. Loosely speaking, some outliers
should be tolerated if this has the benefit of substantially
increasing the margin on the remaining points. We propose a new
boosting algorithm,
-Arc, which allows for the possibility of a pre-specified fraction
of points to lie in the margin area or even on the wrong side of
the decision boundary.
|
