| |
Abstract:
We derive an equivalence between AdaBoost and the dual of a
convex optimization problem, showing that the only difference
between minimizing the exponential loss used by AdaBoost and
maximum likelihood for exponential models is that the latter
requires the model to be normalized to form a conditional
probability distribution over labels. In addition to establishing
a simple and easily understood connection between the two
methods, this framework enables us to derive new regularization
procedures for boosting that directly correspond to penalized
maximum likelihood. Experiments on UCI datasets dupport our
theoretical analysis and give additional insight into the
relationship between boosting and logistic regression.
|
