TY - GEN

T1 - Fast rates for regularized objectives

AU - Sridharan, Karthik

AU - Srebro, Nathan

AU - Shalev-Shwartz, Shai

PY - 2009

Y1 - 2009

N2 - We study convergence properties of empirical minimization of a stochastic strongly convex objective, where the stochastic component is linear. We show that the value attained by the empirical minimizer converges to the optimal value with rate 1/n. The result applies, in particular, to the SVM objective. Thus, we obtain a rate of 1/n on the convergence of the SVM objective (with fixed regularization parameter) to its infinite data limit. We demonstrate how this is essential for obtaining certain type of oracle inequalities for SVMs. The results extend also to approximate minimization as well as to strong convexity with respect to an arbitrary norm, and so also to objectives regularized using other ℓp norms.

AB - We study convergence properties of empirical minimization of a stochastic strongly convex objective, where the stochastic component is linear. We show that the value attained by the empirical minimizer converges to the optimal value with rate 1/n. The result applies, in particular, to the SVM objective. Thus, we obtain a rate of 1/n on the convergence of the SVM objective (with fixed regularization parameter) to its infinite data limit. We demonstrate how this is essential for obtaining certain type of oracle inequalities for SVMs. The results extend also to approximate minimization as well as to strong convexity with respect to an arbitrary norm, and so also to objectives regularized using other ℓp norms.

UR - http://www.scopus.com/inward/record.url?scp=84858785386&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84858785386

SN - 9781605609492

T3 - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

SP - 1545

EP - 1552

BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

PB - Neural Information Processing Systems

T2 - 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008

Y2 - 8 December 2008 through 11 December 2008

ER -