A primal-dual perspective of online learning algorithms

Shai Shalev-Shwartz*, Yoram Singer

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

90 Scopus citations


We describe a novel framework for the design and analysis of online learning algorithms based on the notion of duality in constrained optimization. We cast a sub-family of universal online bounds as an optimization problem. Using the weak duality theorem we reduce the process of online learning to the task of incrementally increasing the dual objective function. The amount by which the dual increases serves as a new and natural notion of progress for analyzing online learning algorithms. We are thus able to tie the primal objective value and the number of prediction mistakes using the increase in the dual.

Original languageAmerican English
Pages (from-to)115-142
Number of pages28
JournalMachine Learning
Issue number2-3
StatePublished - Dec 2007

Bibliographical note

Funding Information:
Acknowledgements Thanks to the anonymous reviewers for helpful comments. This work was supported by the Israeli Science Foundation, grant No. 039-7444.


  • Duality
  • Mistake bounds
  • Online learning
  • Regret bounds


Dive into the research topics of 'A primal-dual perspective of online learning algorithms'. Together they form a unique fingerprint.

Cite this