| |
Abstract:
We replace the commonly used Gaussian noise model in
nonlinear regression by a more flexible noise model based on the
Student
-distribution. The degrees of freedom of the
-distribution can be chosen such that as special cases either the
Gaussian distribution or the Cauchy distribution are realized. The
latter is commonly used in robust regression. Since the
-distribution can be interpreted as being an infinite mixture of
Gaussians, parameters and hyperparameters such as the degrees of
freedom of the
-distribution can be learned from the data based on an EM-learning
algorithm. We show that modeling using the
-distribution leads to improved predictors on real world data
sets. In particular, if outliers are present, the
-distribution is superior to the Gaussian noise model. In effect,
by adapting the degrees of freedom, the system can "learn" to
distinguish between outliers and non-outliers. Especially for
online learning tasks, one is interested in avoiding inappropriate
weight changes due to measurement outliers to maintain stable
online learning capability. We show experimentally that using the
-distribution as a noise model leads to stable online learning
algorithms and outperforms state-of-the art online learning methods
like the extended Kalman filter algorithm.
|
