Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons

Haider, Paul; Ellenberger, Benjamin; Kriener, Laura; Jordan, Jakob; Senn, Walter; Petrovici, Mihai A. (2021). Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons. Advances in Neural Information Processing Systems (NIPS), 35. MIT Press

[img] Text
NeurIPS-2021-latent-equilibrium-arbitrarily-fast-computation-with-arbitrarily-slow-neurons-Paper.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (2MB) | Request a copy

The response time of physical computational elements is finite, and neurons are no exception. In hierarchical models of cortical networks each layer thus introduces a response lag. This inherent property of physical dynamical systems results in delayed processing of stimuli and causes a timing mismatch between network output and instructive signals, thus afflicting not only inference, but also learning. We introduce Latent Equilibrium, a new framework for inference and learning in networks of slow components which avoids these issues by harnessing the ability of biological neurons to phase-advance their output with respect to their membrane potential. This principle enables quasi-instantaneous inference independent of network depth and avoids the need for phased plasticity or computationally expensive network relaxation phases. We jointly derive disentangled neuron and synapse dynamics from a prospective energy function that depends on a network's generalized position and momentum. The resulting model can be interpreted as a biologically plausible approximation of error backpropagation in deep cortical networks with continuous-time, leaky neuronal dynamics and continuously active, local plasticity. We demonstrate successful learning of standard benchmark datasets, achieving competitive performance using both fully-connected and convolutional architectures, and show how our principle can be applied to detailed models of cortical microcircuitry. Furthermore, we study the robustness of our model to spatio-temporal substrate imperfections to demonstrate its feasibility for physical realization, be it in vivo or in silico.

Item Type:

Conference or Workshop Item (Paper)


04 Faculty of Medicine > Pre-clinic Human Medicine > Institute of Physiology

Graduate School:

Graduate School for Cellular and Biomedical Sciences (GCB)

UniBE Contributor:

Haider, Paul, Ellenberger, Benjamin Till, Kriener, Laura Magdalena, Jordan, Jakob Jürgen, Senn, Walter, Petrovici, Mihai Alexandru


600 Technology > 610 Medicine & health




MIT Press




Virginie Sabado

Date Deposited:

20 Apr 2022 13:26

Last Modified:

05 Dec 2022 16:18




Actions (login required)

Edit item Edit item
Provide Feedback