Gradient estimation in dendritic reinforcement learning

Schiess, Mathieu; Urbanzik, Robert; Senn, Walter (2012). Gradient estimation in dendritic reinforcement learning. Journal of mathematical neuroscience, 2(2), pp. 1-19. London: BioMed Central 10.1186/2190-8567-2-2

Preview

Text 2190-8567-2-2.pdf - Published Version Available under License Creative Commons: Attribution (CC-BY). Download (629kB) | Preview

We study synaptic plasticity in a complex neuronal cell model where NMDA-spikes can arise in certain dendritic zones. In the context of reinforcement learning, two kinds of plasticity rules are derived, zone reinforcement (ZR) and cell reinforcement (CR), which both optimize the expected reward by stochastic gradient ascent. For ZR, the synaptic plasticity response to the external reward signal is modulated exclusively by quantities which are local to the NMDA-spike initiation zone in which the synapse is situated. CR, in addition, uses nonlocal feedback from the soma of the cell, provided by mechanisms such as the backpropagating action potential. Simulation results show that, compared to ZR, the use of nonlocal feedback in CR can drastically enhance learning performance. We suggest that the availability of nonlocal feedback for learning is a key advantage of complex neurons over networks of simple point neurons, which have previously been found to be largely equivalent with regard to computational capability.

Interest & Impact

Downloads

254 since deposited on 04 Oct 2013
15 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item