From average case complexity to improper learning complexity

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

63 Scopus citations


The basic problem in the PAC model of computational learning theory is to determine which hypothesis classes are efficiently learnable. There is presently a dearth of results showing hardness of learning problems. Moreover, the existing lower bounds fall short of the best known algorithms. The biggest challenge in proving complexity results is to establish hardness of improper learning (a.k.a. representation independent learning). The dificulty in proving lower bounds for improper learning is that the standard reductions from NP-hard problems do not seem to apply in this context. There is essentially only one known approach to proving lower bounds on improper learning. It was initiated in [21] and relies on cryptographic assumptions. We introduce a new technique for proving hardness of improper learning, based on reductions from problems that are hard on average. We put forward a (fairly strong) generalization of Feige's assumption [13] about the complexity of refuting random constraint satisfaction problems. Combining this assumption with our new technique yields far reaching implications. In particular, • Learning DNF's is hard. • Agnostically learning halfspaces with a constant approximation ratio is hard. • Learning an intersection of ω(1) halfspaces is hard.

Original languageAmerican English
Title of host publicationSTOC 2014 - Proceedings of the 2014 ACM Symposium on Theory of Computing
PublisherAssociation for Computing Machinery
Number of pages8
ISBN (Print)9781450327107
StatePublished - 2014
Event4th Annual ACM Symposium on Theory of Computing, STOC 2014 - New York, NY, United States
Duration: 31 May 20143 Jun 2014

Publication series

NameProceedings of the Annual ACM Symposium on Theory of Computing
ISSN (Print)0737-8017


Conference4th Annual ACM Symposium on Theory of Computing, STOC 2014
Country/TerritoryUnited States
CityNew York, NY


  • Average case complexity
  • CSP problems
  • DNFs
  • Halfspaces
  • Hardness of improper learning
  • Resolution lower bounds


Dive into the research topics of 'From average case complexity to improper learning complexity'. Together they form a unique fingerprint.

Cite this