Learnability, stability and uniform convergence

Shai Shalev-Shwartz*, Ohad Shamir, Nathan Srebro, Karthik Sridharan

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

278 Scopus citations


The problem of characterizing learnability is the most basic question of statistical learning theory. A fundamental and long-standing answer, at least for the case of supervised classification and regression, is that learnability is equivalent to uniform convergence of the empirical risk to the population risk, and that if a problem is learnable, it is learnable via empirical risk minimization. In this paper, we consider the General Learning Setting (introduced by Vapnik), which includes most statistical learning problems as special cases. We show that in this setting, there are non-trivial learning problems where uniform convergence does not hold, empirical risk minimization fails, and yet they are learnable using alternative mechanisms. Instead of uniform convergence, we identify stability as the key necessary and sufficient condition for learnability. Moreover, we show that the conditions for learnability in the general setting are significantly more complex than in supervised classification and regression.

Original languageAmerican English
Pages (from-to)2635-2670
Number of pages36
JournalJournal of Machine Learning Research
StatePublished - Oct 2010


  • Learnability
  • Stability
  • Statistical learning theory
  • Stochastic convex optimization
  • Uniform convergence


Dive into the research topics of 'Learnability, stability and uniform convergence'. Together they form a unique fingerprint.

Cite this