Complexity theoretic limitations on learning halfspaces

Amit Daniely*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

68 Scopus citations

Abstract

We study the problem of agnostically learning halfspaces which is defined by a fixed but unknown distribution D on ℚn × {±1}. We define ErrHALF(D) as the least error of a halfspace classifier for D. A learner who can access D has to return a hypothesis whose error is small compared to ErrHALF(D). Using the recently developed method of Daniely, Linial and Shalev-Shwartz we prove hardness of learning results assuming that random K-XOR formulas are hard to (strongly) refute. We show that no efficient learning algorithm has nontrivial worst-case performance even under the guarantees that ErrHALp(D) ≤ η for arbitrarily small constant η > 0, and that D is supported in {±1 }n × {±1}. Namely, even under these favorable conditions, and for every c > 0, it is hard to return a hypothesis with error ≤ 1/2/1/nc. In particular, no efficient algorithm can achieve a constant approximation ratio. Under a stronger version of the assumption (where K can be poly-logarithmic in n), we can take η = 2-log1-v(n) for arbitrarily small ? > 0. These results substantially improve on previously known results, that only show hardness of exact learning.

Original languageAmerican English
Title of host publicationSTOC 2016 - Proceedings of the 48th Annual ACM SIGACT Symposium on Theory of Computing
EditorsYishay Mansour, Daniel Wichs
PublisherAssociation for Computing Machinery
Pages105-117
Number of pages13
ISBN (Electronic)9781450341325
DOIs
StatePublished - 19 Jun 2016
Externally publishedYes
Event48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016 - Cambridge, United States
Duration: 19 Jun 201621 Jun 2016

Publication series

NameProceedings of the Annual ACM Symposium on Theory of Computing
Volume19-21-June-2016
ISSN (Print)0737-8017

Conference

Conference48th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2016
Country/TerritoryUnited States
CityCambridge
Period19/06/1621/06/16

Bibliographical note

Publisher Copyright:
© 2016 ACM.

Keywords

  • Halfspaces
  • Hardness of learning
  • Random XOR

Fingerprint

Dive into the research topics of 'Complexity theoretic limitations on learning halfspaces'. Together they form a unique fingerprint.

Cite this