TY - GEN

T1 - Learning kernel-based halfspaces with the zero-one loss

AU - Shalev-Shwartz, Shai

AU - Shamir, Ohad

AU - Sridharan, Karthik

PY - 2010

Y1 - 2010

N2 - We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the zero-one loss function. Unlike most previous formulations which rely on surrogate convex loss functions (e.g. hinge-loss in SVM and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural zero-one loss function. The proposed algorithm can learn kernel-based halfspaces in worst-case time poly (exp (L log(L/ε))), for any distribution, where L is a Lipschitz constant (which can be thought of as the reciprocal of the margin), and the learned classifier is worse than the optimal halfspace by at most ε. We also prove a hardness result, showing that under a certain cryptographic assumption, no algorithm can learn kernel-based halfspaces in time polynomial in L.

AB - We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the zero-one loss function. Unlike most previous formulations which rely on surrogate convex loss functions (e.g. hinge-loss in SVM and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural zero-one loss function. The proposed algorithm can learn kernel-based halfspaces in worst-case time poly (exp (L log(L/ε))), for any distribution, where L is a Lipschitz constant (which can be thought of as the reciprocal of the margin), and the learned classifier is worse than the optimal halfspace by at most ε. We also prove a hardness result, showing that under a certain cryptographic assumption, no algorithm can learn kernel-based halfspaces in time polynomial in L.

UR - http://www.scopus.com/inward/record.url?scp=84898062142&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84898062142

SN - 9780982252925

T3 - COLT 2010 - The 23rd Conference on Learning Theory

SP - 441

EP - 450

BT - COLT 2010 - The 23rd Conference on Learning Theory

T2 - 23rd Conference on Learning Theory, COLT 2010

Y2 - 27 June 2010 through 29 June 2010

ER -