Skip to main content

Advertisement

Stochastic gradient ascent learning with spike timing dependent plasticity

Article metrics

  • 894 Accesses

Stochastic gradient ascent learning exploits correlations of parameter variations with overall success of a system. This algorithmic idea has been related to neuronal network learning by postulating eligibility traces at synapses, which make them selectable for synaptic changes depending on later reward signals ([1] and [2]). Formalizations of the synaptic and neuronal dynamics supporting gradient ascent learning in terms of differential equations exhibit strong similarities with a recent formulation of spike timing dependent plasticity (STDP) [3] when it is combined with a reward signal. Here we present conditions under which reward modulated STDP is in fact guaranteed to maximize expected reward. We present numerical simulations underlining the relevance of realistic STDP models for reward dependent learning. In particular, we find that the nonlinear adaptation to pre- and post-synaptic activities of STDP [3] contributes to stable learning.

Figure 1
figure1

Learning the XOR function with a reward modulated STDP rule. Left: Output activity versus training episode in a feed forward network with Poisson-like neurons (2 input nodes, 10 hidden nodes and 1 output node). The output activity for the [true, false] and [false, true] inputs becomes stronger, while the output for the [true, true] and [false, false] inputs becomes weak after training. Right: Accumulated administered reward for the four input patterns versus training episode.

References

  1. 1.

    Sebastian Seung H: Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission. Neuron. 2003, 40 (6): 1063-1073. 10.1016/S0896-6273(03)00761-X.

  2. 2.

    Xiaohui Xie, Sebastian Seung H: Learning in neural networks by reinforcement of irregular spiking. Phys Rev E. 2004, 69 (4): 041909-1-041909-10.

  3. 3.

    Schmiedt Joscha T, Christian Albers, Klaus Pawelzik: Spike timing-dependent plasticity as dynamic filter. Advances in Neural Information Processing Systems 23. Edited by: J. Lafferty and C. K. I. Williams and J. Shawe-Taylor and R.S. Zemel and A. Culotta. 2010, 2110-2118.

Download references

Author information

Correspondence to Joana Vieira.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Vieira, J., Arévalo, O. & Pawelzik, K. Stochastic gradient ascent learning with spike timing dependent plasticity. BMC Neurosci 12, P250 (2011) doi:10.1186/1471-2202-12-S1-P250

Download citation

Keywords

  • Differential Equation
  • Animal Model
  • Algorithmic Idea
  • Parameter Variation
  • Neuronal Network