Smooth ε-insensitive regression by loss symmetrization

Ofer Dekel*, Shai Shalev-Shwartz, Yoram Singer

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

21 Scopus citations


We describe new loss functions for regression problems along with an accompanying algorithmic framework which utilizes these functions. These loss functions are derived by symmetrization of margin-based losses commonly used in boosting algorithms, namely, the logistic loss and the exponential loss. The resulting symmetric logistic loss can be viewed as a smooth approximation to the ε-insensitive hinge loss used in support vector regression. We describe and analyze two parametric families of batch learning algorithms for minimizing these symmetric losses. The first family employs an iterative log-additive update which can be viewed as a regression counterpart to recent boosting algorithms. The second family utilizes an iterative additive update step. We also describe and analyze online gradient descent (GD) and exponentiated gradient (EG) algorithms for the symmetric logistic loss. A byproduct of our work is a new simple form of regularization for boosting-based classification and regression algorithms. Our regression framework also has implications on classification algorithms, namely, a new additive update boosting algorithm for classification. We demonstrate the merits of our algorithms in a series of experiments.

Original languageAmerican English
JournalJournal of Machine Learning Research
StatePublished - 2005


Dive into the research topics of 'Smooth ε-insensitive regression by loss symmetrization'. Together they form a unique fingerprint.

Cite this