Most ReLU networks suffer from l2 adversarial perturbations

Amit Daniely, Hadas Schacham

Research output: Contribution to journalConference articlepeer-review

5 Scopus citations

Abstract

We consider ReLU networks with random weights, in which the dimension decreases at each layer. We show that for most such networks, most examples x admit an adversarial perturbation at an Euclidean distance of O (kvxdk) , where d is the input dimension. Moreover, this perturbation can be found via gradient flow, as well as gradient descent with sufficiently small steps. This result can be seen as an explanation to the abundance of adversarial examples, and to the fact that they are found via gradient descent.

Original languageAmerican English
JournalAdvances in Neural Information Processing Systems
Volume2020-December
StatePublished - 2020
Event34th Conference on Neural Information Processing Systems, NeurIPS 2020 - Virtual, Online
Duration: 6 Dec 202012 Dec 2020

Bibliographical note

Publisher Copyright:
© 2020 Neural information processing systems foundation. All rights reserved.

Cite this