Efficient Learning of CNNs using Patch Based Features

Alon Brutzkus, Amir Globerson, Eran Malach*, Alon Regev Netser*, Shai Shalev-Shwartz

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

6 Scopus citations

Abstract

Recent work has demonstrated the effectiveness of using patch based representations when learning from image data. Here we provide theoretical support for this observation, by showing that a simple semi-supervised algorithm that uses patch statistics can efficiently learn labels produced by a one-hidden-layer Convolutional Neural Network (CNN). Since CNNs are known to be computationally hard to learn in the worst case, our analysis holds under some distributional assumptions. We show that these assumptions are necessary and sufficient for our results to hold. We verify that the distributional assumptions hold on real-world data by experimenting on the CIFAR-10 dataset, and find that the analyzed algorithm outperforms a vanilla one-hidden-layer CNN. Finally, we demonstrate that by running the algorithm in a layer-by-layer fashion we can build a deep model which gives further improvements, hinting that this method provides insights about the behavior of deep CNNs.

Original languageAmerican English
Pages (from-to)2336-2356
Number of pages21
JournalProceedings of Machine Learning Research
Volume162
StatePublished - 2022
Event39th International Conference on Machine Learning, ICML 2022 - Baltimore, United States
Duration: 17 Jul 202223 Jul 2022
https://proceedings.mlr.press/v162/

Bibliographical note

Funding Information:
This project was partially funded by the European Research Council (ERC) under the European Unions Horizon 2020 research and innovation programme (grant ERC HOLI 819080). AB is supported by a Google PhD fellowship.

Publisher Copyright:
Copyright © 2022 by the author(s)

Fingerprint

Dive into the research topics of 'Efficient Learning of CNNs using Patch Based Features'. Together they form a unique fingerprint.

Cite this