Learning Only When Necessary: Better Memories of Correlated Patterns in Networks with Bounded Synapses

Senn, Walter; Fusi, Stefano (2005). Learning Only When Necessary: Better Memories of Correlated Patterns in Networks with Bounded Synapses. Neural computation, 17(10), pp. 2106-2138. MIT Press 10.1162/0899766054615644

Full text not available from this repository.

Learning in a neuronal network is often thought of as a linear superposition of synaptic modifications induced by individual stimuli. However, since biological synapses are naturally bounded, a linear superposition would cause fast forgetting of previously acquired memories. Here we show that this forgetting can be avoided by introducing additional constraints on the synaptic and neural dynamics. We consider Hebbian plasticity of excitatory synapses. A synapse is modified only if the postsynaptic response does not match the desired output. With this learning rule, the original memory performances with unbounded weights are regained, provided that (1) there is some global inhibition, (2) the learning rate is small, and (3) the neurons can discriminate small differences in the total synaptic input (e.g., by making the neuronal threshold small compared to the total postsynaptic input). We prove in the form of a generalized perceptron convergence theorem that under these constraints, a neuron learns to classify any linearly separable set of patterns, including a wide class of highly correlated random patterns. During the learning process, excitation becomes roughly balanced by inhibition, and the neuron classifies the patterns on the basis of small differences around this balance. The fact that synapses saturate has the additional benefit that nonlinearly separable patterns, such as similar patterns with contradicting outputs, eventually generate a subthreshold response, and therefore silence neurons that cannot provide any information.

Item Type:	Journal Article (Original Article)
Division/Institute:	04 Faculty of Medicine > Pre-clinic Human Medicine > Institute of Physiology
UniBE Contributor:	Senn, Walter, Fusi, Stefano
Subjects:	600 Technology > 610 Medicine & health
ISSN:	0899-7667
Publisher:	MIT Press
Language:	English
Submitter:	Virginie Sabado
Date Deposited:	18 Jan 2023 15:56
Last Modified:	18 Jan 2023 23:28
Publisher DOI:	10.1162/0899766054615644
PubMed ID:	16105220
URI:	https://boris.unibe.ch/id/eprint/177104