COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS

Eran Malach; Shai Shalev-Shwartz

COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS

Eran Malach, Shai Shalev-Shwartz

The Rachel and Selim Benin School of Engineering and Computer Science

Research output: Contribution to conference › Paper › peer-review

7 Scopus citations

Abstract

Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.

Original language	English
State	Published - 2021
Event	9th International Conference on Learning Representations, ICLR 2021 - Virtual, Online, Austria Duration: 3 May 2021 → 7 May 2021

Conference

Conference	9th International Conference on Learning Representations, ICLR 2021
Country/Territory	Austria
City	Virtual, Online
Period	3/05/21 → 7/05/21

Bibliographical note

Publisher Copyright:
© 2021 ICLR 2021 - 9th International Conference on Learning Representations. All rights reserved.

Cite this

@conference{ecf853d5ddc74d5885b28ffe6be1a2c6,

title = "COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS",

abstract = "Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.",

author = "Eran Malach and Shai Shalev-Shwartz",

note = "Publisher Copyright: {\textcopyright} 2021 ICLR 2021 - 9th International Conference on Learning Representations. All rights reserved.; 9th International Conference on Learning Representations, ICLR 2021 ; Conference date: 03-05-2021 Through 07-05-2021",

year = "2021",

language = "אנגלית",

}

TY - CONF

T1 - COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS

AU - Malach, Eran

AU - Shalev-Shwartz, Shai

PY - 2021

Y1 - 2021

N2 - Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.

AB - Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.

UR - https://www.scopus.com/pages/publications/85102166334

M3 - ???researchoutput.researchoutputtypes.contributiontoconference.paper???

AN - SCOPUS:85102166334

T2 - 9th International Conference on Learning Representations, ICLR 2021

Y2 - 3 May 2021 through 7 May 2021

ER -

COMPUTATIONAL SEPARATION BETWEEN CONVOLUTIONAL AND FULLY-CONNECTED NETWORKS

Abstract

Conference

Bibliographical note

Other files and links

Fingerprint

Cite this