From Local Pseudorandom Generators to Hardness of Learning

Amit Daniely, Gal Vardi

Research output: Contribution to journalConference articlepeer-review

10 Scopus citations

Abstract

We prove hardness-of-learning results under a well-studied assumption on the existence of local pseudorandom generators. As we show, this assumption allows us to surpass the current state of the art, and prove hardness of various basic problems, with no hardness results to date. Our results include: hardness of learning shallow ReLU neural networks under the Gaussian distribution and other distributions; hardness of learning intersections of ω(1) halfspaces, DNF formulas with ω(1) terms, and ReLU networks with ω(1) hidden neurons; hardness of weakly learning deterministic finite automata under the uniform distribution; hardness of weakly learning depth-3 Boolean circuits under the uniform distribution, as well as distribution-specific hardness results for learning DNF formulas and intersections of halfspaces. We also establish lower bounds on the complexity of learning intersections of a constant number of halfspaces, and ReLU networks with a constant number of hidden neurons. Moreover, our results imply the hardness of virtually all improper PAC-learning problems (both distribution-free and distribution-specific) that were previously shown hard under other assumptions.

Original languageAmerican English
Pages (from-to)1358-1394
Number of pages37
JournalProceedings of Machine Learning Research
Volume134
StatePublished - 2021
Event34th Conference on Learning Theory, COLT 2021 - Boulder, United States
Duration: 15 Aug 202119 Aug 2021

Bibliographical note

Funding Information:
We thank Benny Applebaum and anonymous reviewers for their valuable comments. This research is partially supported by ISF grant 2258/19.

Publisher Copyright:
© 2021 A. Daniely & G. Vardi.

Fingerprint

Dive into the research topics of 'From Local Pseudorandom Generators to Hardness of Learning'. Together they form a unique fingerprint.

Cite this