COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS

Eran Malach, Shai Shalev-Shwartz

Research output: Contribution to conferencePaperpeer-review

6 Scopus citations

Abstract

Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.

Original languageAmerican English
StatePublished - 2021
Event9th International Conference on Learning Representations, ICLR 2021 - Virtual, Online
Duration: 3 May 20217 May 2021

Conference

Conference9th International Conference on Learning Representations, ICLR 2021
CityVirtual, Online
Period3/05/217/05/21

Bibliographical note

Publisher Copyright:
© 2021 ICLR 2021 - 9th International Conference on Learning Representations. All rights reserved.

Fingerprint

Dive into the research topics of 'COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS'. Together they form a unique fingerprint.

Cite this