Stochastic methods for l1-regularized loss minimization

Shai Shalev-Shwartz*, Ambuj Tewari

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

217 Scopus citations

Abstract

We describe and analyze two stochastic methods for l1 regularized loss minimization problems, such as the Lasso. The first method updates the weight of a single feature at each iteration while the second method updates the entire weight vector but only uses a single training example at each iteration. In both methods, the choice of feature or example is uniformly at random. Our theoretical runtime analysis suggests that the stochastic methods should outperform state-of-the-art deterministic approaches, including their deterministic counterparts, when the size of the problem is large. We demonstrate the advantage of stochastic methods by experimenting with synthetic and natural data sets.1.

Original languageAmerican English
Pages (from-to)1865-1892
Number of pages28
JournalJournal of Machine Learning Research
Volume12
StatePublished - Jun 2011

Keywords

  • Coordinate descent
  • L1 regularization
  • Mirror descent
  • Optimization
  • Sparsity

Fingerprint

Dive into the research topics of 'Stochastic methods for l1-regularized loss minimization'. Together they form a unique fingerprint.

Cite this