TY - JOUR
T1 - Learning kernel-based halfspaces with the 0-1 loss
AU - Shalev-Shwartz, Shai
AU - Shamir, Ohad
AU - Sridharan, Karthik
PY - 2011
Y1 - 2011
N2 - We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the 0-1 loss function. Unlike most of the previous formulations, which rely on surrogate convex loss functions (e.g., hinge-loss in support vector machines (SVMs) and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural 0-1 loss function. The proposed algorithm can learn kernel-based halfspaces in worst-case time poly(exp(Llog(L/ε))), for any distribution, where L is a Lipschitz constant (which can be thought of as the reciprocal of the margin), and the learned classifier is worse than the optimal halfspace by at most ε. We also prove a hardness result, showing that under a certain cryptographic assumption, no algorithm can learn kernel-based halfspaces in time polynomial in L.
AB - We describe and analyze a new algorithm for agnostically learning kernel-based halfspaces with respect to the 0-1 loss function. Unlike most of the previous formulations, which rely on surrogate convex loss functions (e.g., hinge-loss in support vector machines (SVMs) and log-loss in logistic regression), we provide finite time/sample guarantees with respect to the more natural 0-1 loss function. The proposed algorithm can learn kernel-based halfspaces in worst-case time poly(exp(Llog(L/ε))), for any distribution, where L is a Lipschitz constant (which can be thought of as the reciprocal of the margin), and the learned classifier is worse than the optimal halfspace by at most ε. We also prove a hardness result, showing that under a certain cryptographic assumption, no algorithm can learn kernel-based halfspaces in time polynomial in L.
KW - Kernel methods
KW - Learning halfspaces
KW - Learning theory
UR - http://www.scopus.com/inward/record.url?scp=84855575451&partnerID=8YFLogxK
U2 - 10.1137/100806126
DO - 10.1137/100806126
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
AN - SCOPUS:84855575451
SN - 0097-5397
VL - 40
SP - 1623
EP - 1646
JO - SIAM Journal on Computing
JF - SIAM Journal on Computing
IS - 6
ER -