Fast rates for regularized objectives

Karthik Sridharan*, Nathan Srebro, Shai Shalev-Shwartz

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

83 Scopus citations

Abstract

We study convergence properties of empirical minimization of a stochastic strongly convex objective, where the stochastic component is linear. We show that the value attained by the empirical minimizer converges to the optimal value with rate 1/n. The result applies, in particular, to the SVM objective. Thus, we obtain a rate of 1/n on the convergence of the SVM objective (with fixed regularization parameter) to its infinite data limit. We demonstrate how this is essential for obtaining certain type of oracle inequalities for SVMs. The results extend also to approximate minimization as well as to strong convexity with respect to an arbitrary norm, and so also to objectives regularized using other ℓp norms.

Original languageAmerican English
Title of host publicationAdvances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
PublisherNeural Information Processing Systems
Pages1545-1552
Number of pages8
ISBN (Print)9781605609492
StatePublished - 2009
Externally publishedYes
Event22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 - Vancouver, BC, Canada
Duration: 8 Dec 200811 Dec 2008

Publication series

NameAdvances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

Conference

Conference22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
Country/TerritoryCanada
CityVancouver, BC
Period8/12/0811/12/08

Cite this