Capacity of neural networks with discrete synaptic couplings

The authors study the optimal storage capacity of neural networks with discrete local constraints on the synaptic couplings J_ij. Models with such constraints include those with binary couplings J_ij=+or-1 or J_ij=0, 1, quantised couplings with larger synaptic range, e.g. J _ij=+or-1/L, +or-2/L, . . ., +or-1 and, in the limit, continuous couplings confined to the hypercube mod J_ij mod <or=1 ('box confinement'). They find that the optimal storage capacity alpha ( kappa ) is best determined by the vanishing of a suitably defined 'entropy' as calculated in the replica symmetric approximation. They also extend their results to cases with biased memories and make contact with sparse coding models.