Most Neural Networks Are Almost Learnable

Amit Daniely; Nathan Srebro; Gal Vardi

Most Neural Networks Are Almost Learnable

Amit Daniely, Nathan Srebro, Gal Vardi

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Contribution to journal › Conference article › peer-review

Abstract

We present a PTAS for learning random constant-depth networks. We show that for any fixed ϵ > 0 and depth i, there is a poly-time algorithm that for any distribution on √d· S^d−1 learns random Xavier networks of depth i, up to an additive error of ϵ. The algorithm runs in time and sample complexity of (d̄)^poly(ϵ−1), where d̄is the size of the network. For some cases of sigmoid and ReLU-like activations the bound can be improved to (d̄)^{polylog(ϵ−1)}, resulting in a quasi-poly-time algorithm for learning constant depth random networks.

Original language	English
Journal	Advances in Neural Information Processing Systems
Volume	36
State	Published - 2023
Event	37th Conference on Neural Information Processing Systems, NeurIPS 2023 - New Orleans, United States Duration: 10 Dec 2023 → 16 Dec 2023 Conference number: 37

Bibliographical note

Publisher Copyright:
© 2023 Neural information processing systems foundation. All rights reserved.

Cite this

@article{6537489a11d74b3dbb8a98ab83c4e7b0,

title = "Most Neural Networks Are Almost Learnable",

abstract = "We present a PTAS for learning random constant-depth networks. We show that for any fixed ϵ > 0 and depth i, there is a poly-time algorithm that for any distribution on √d· Sd−1 learns random Xavier networks of depth i, up to an additive error of ϵ. The algorithm runs in time and sample complexity of ({\=d})poly(ϵ−1), where {\=d}is the size of the network. For some cases of sigmoid and ReLU-like activations the bound can be improved to ({\=d})polylog(ϵ−1), resulting in a quasi-poly-time algorithm for learning constant depth random networks.",

author = "Amit Daniely and Nathan Srebro and Gal Vardi",

note = "Publisher Copyright: {\textcopyright} 2023 Neural information processing systems foundation. All rights reserved.; 37th Conference on Neural Information Processing Systems, NeurIPS 2023 ; Conference date: 10-12-2023 Through 16-12-2023",

year = "2023",

language = "אנגלית",

volume = "36",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

}

TY - JOUR

T1 - Most Neural Networks Are Almost Learnable

AU - Daniely, Amit

AU - Srebro, Nathan

AU - Vardi, Gal

N1 - Conference code: 37

PY - 2023

Y1 - 2023

N2 - We present a PTAS for learning random constant-depth networks. We show that for any fixed ϵ > 0 and depth i, there is a poly-time algorithm that for any distribution on √d· Sd−1 learns random Xavier networks of depth i, up to an additive error of ϵ. The algorithm runs in time and sample complexity of (d̄)poly(ϵ−1), where d̄is the size of the network. For some cases of sigmoid and ReLU-like activations the bound can be improved to (d̄)polylog(ϵ−1), resulting in a quasi-poly-time algorithm for learning constant depth random networks.

AB - We present a PTAS for learning random constant-depth networks. We show that for any fixed ϵ > 0 and depth i, there is a poly-time algorithm that for any distribution on √d· Sd−1 learns random Xavier networks of depth i, up to an additive error of ϵ. The algorithm runs in time and sample complexity of (d̄)poly(ϵ−1), where d̄is the size of the network. For some cases of sigmoid and ReLU-like activations the bound can be improved to (d̄)polylog(ϵ−1), resulting in a quasi-poly-time algorithm for learning constant depth random networks.

UR - http://www.scopus.com/inward/record.url?scp=85191175143&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.conferencearticle???

AN - SCOPUS:85191175143

SN - 1049-5258

VL - 36

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

T2 - 37th Conference on Neural Information Processing Systems, NeurIPS 2023

Y2 - 10 December 2023 through 16 December 2023

ER -

Most Neural Networks Are Almost Learnable

Abstract

Bibliographical note

Other files and links

Fingerprint

Cite this