TY - JOUR
T1 - Memoryless Optimality
T2 - Neurons Do Not Need Adaptation to Optimally Encode Stimuli With Arbitrarily Complex Statistics
AU - Forkosh, Oren
N1 - DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.
PY - 2022/12
Y1 - 2022/12
N2 - Our neurons seem capable of handling any type of data, regardless of its scale or statistical properties. In this letter, we suggest that optimal coding may occur at the single-neuron level without requiring memory, adaptation, or evolutionary-driven fit to the stimuli. We refer to a neural circuit as optimal if it maximizes the mutual information between its inputs and outputs. We show that often encountered differentiator neurons, or neurons that respond mainly to changes in the input, are capable of using all their information capacity when handling samples of any statistical distribution. We demonstrate this optimality using both analytical methods and simulations. In addition to demonstrating the simplicity and elegance of neural processing, this result might provide a way to improve the handling of data by artificial neural networks.
AB - Our neurons seem capable of handling any type of data, regardless of its scale or statistical properties. In this letter, we suggest that optimal coding may occur at the single-neuron level without requiring memory, adaptation, or evolutionary-driven fit to the stimuli. We refer to a neural circuit as optimal if it maximizes the mutual information between its inputs and outputs. We show that often encountered differentiator neurons, or neurons that respond mainly to changes in the input, are capable of using all their information capacity when handling samples of any statistical distribution. We demonstrate this optimality using both analytical methods and simulations. In addition to demonstrating the simplicity and elegance of neural processing, this result might provide a way to improve the handling of data by artificial neural networks.
UR - http://www.scopus.com/inward/record.url?scp=85141352298&partnerID=8YFLogxK
U2 - 10.1162/neco_a_01543
DO - 10.1162/neco_a_01543
M3 - Letter
C2 - 36283043
AN - SCOPUS:85141352298
SN - 0899-7667
VL - 34
SP - 2374
EP - 2387
JO - Neural Computation
JF - Neural Computation
IS - 12
M1 - 12
ER -