Expression of Fractals Through Neural Network Functions.

N. Dym, B. Sober, I. Daubechies

Research output: Contribution to journalArticlepeer-review

Abstract

To help understand the underlying mechanisms of neural networks (NNs), several groups have studied the number of linear regions $ of piecewise linear (PwL) functions, generated by deep neural networks (DNN). In particular, they showed that $ can grow exponentially with the number of network parameters $p$ , a property often used to explain the advantages of deep over shallow NNs. Nonetheless, a dimension argument shows that DNNs cannot generate all PwL functions with $ linear regions when $ p$ . It is thus natural to seek to characterize specific families of functions with $ p$ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) recursively construct a sequence of PwL functions $F_k$ with a number of linear regions which is exponential in $k$ . We show that $F_k$ can be generated by a NN using only $O(k)$ parameters. IFS are used extensively to generate natural-looking landscape textures in artificial images as w
Original languageEnglish
Pages (from-to)57 - 66
JournalIEEE Journal on Selected Areas in Information Theory, Selected Areas in Information Theory, IEEE Journal on, IEEE J. Sel. Areas Inf. Theory
Volume1
Issue number1
StatePublished - 2020

Keywords

  • Communication
  • Networking and Broadcast Technologies
  • Fractals
  • Artificial neural networks
  • Data structures
  • Boolean functions
  • Image coding
  • Neural networks expressive power
  • fractals
  • iterated function systems
  • neural network functions

Fingerprint

Dive into the research topics of 'Expression of Fractals Through Neural Network Functions.'. Together they form a unique fingerprint.

Cite this