Computational Separation Between Convolutional and Fully-Connected Networks.

Eran Malach, Shai Shalev-Shwartz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Convolutional neural networks (CNN) exhibit unmatched performance in a multitude of computer vision tasks. However, the advantage of using convolutional networks over fully-connected networks is not understood from a theoretical perspective. In this work, we show how convolutional networks can leverage locality in the data, and thus achieve a computational advantage over fully-connected networks. Specifically, we show a class of problems that can be efficiently solved using convolutional networks trained with gradient-descent, but at the same time is hard to learn using a polynomial-size fully-connected network.
Original languageEnglish
Title of host publicationICLR 2021
Number of pages12
StatePublished - 2021
EventInternational Conference on Learning Representations, ICLR 2021 - Vienna, Austria
Duration: 3 May 20217 May 2021
Conference number: 9


ConferenceInternational Conference on Learning Representations, ICLR 2021
Internet address


  • Neural networks
  • Deep learning
  • Convolutional networks
  • Fully-Connected Networks
  • Gradient descent


Dive into the research topics of 'Computational Separation Between Convolutional and Fully-Connected Networks.'. Together they form a unique fingerprint.

Cite this