Spatio-temporal credit assignment in population learning

Friedrich, Johannes; Urbanczik, Robert; Senn, Walter (2010). Spatio-temporal credit assignment in population learning. In: 6th Clinical Neuroscience Meeting Bern. Bern, Switzerland. November 29th 2010.

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

Item Type:

Conference or Workshop Item (Speech)

Division/Institute:

04 Faculty of Medicine > Pre-clinic Human Medicine > Institute of Physiology

UniBE Contributor:

Friedrich, Johannes, Urbanczik, Robert, Senn, Walter

Subjects:

600 Technology > 610 Medicine & health

Language:

English

Submitter:

Factscience Import

Date Deposited:

04 Oct 2013 14:11

Last Modified:

05 Dec 2022 14:01

URI:

https://boris.unibe.ch/id/eprint/1915 (FactScience: 203996)

Actions (login required)

Edit item Edit item
Provide Feedback