This makes it very unlikely that BPTT is used by the brain 5. But the dilemma is that BPTT requires storing the intermediate states of all neurons during a network computation, and merging these in a subsequent offline process with gradients that are computed backwards in time (see Fig. With a careful choice of the pseudo derivative for handling the discontinuous dynamics of spiking neurons, one can apply BPTT also to RSNNs, and RSNNs were able to learn in this way to solve really demanding computational tasks 3, 4.

These are combined with online learning signals according to Eq.

Feedforward computation of eligibility traces is indicated in blue. c Loss gradients of BPTT are propagated backwards in time and retrograde across synapses in an offline manner, long after the forward computation has passed a layer.

All synapses in this array have the same weight: the weight of this synaptic connection in the RSNN. A synaptic connection from neuron i to neuron j of the RSNN is replaced by an array of feedforward connections, one for each time step t, which goes from the copy of neuron i in the layer for time step t to a copy of neuron j in the layer for time step t + 1. It has a new copy of the neurons of the RSNN for each time step t. b BPTT computes gradients in the unrolled version of the network. Output neurons y provide a low-pass filter of a weighted sum of network spikes z. This method is called backpropagation through time (BPTT), as it requires propagation of gradients backwards in time.Ī RSNN with network inputs x, neuron spikes z, hidden neuron states h, and output targets y *, for each time step t of the RSNN computation. 1b, and applying the backpropagation algorithm to that (Fig. In machine learning, one trains recurrent neural networks by unrolling the network into a virtual feedforward network 1, see Fig. Because of this functional relation to LSTM networks these RSNN models are referred to as LSNNs 3. In fact, RSNNs trained through gradient descent acquire then similar computing capabilities as networks of LSTM (long short-term memory) units, the state of the art for recurrent neural networks in machine learning. Neurons with SFA are quite common in the neocortex 2, and it turns out that their inclusion in the RSNN significantly increases the computational power of the network 3. This problem is aggravated if there are slowly changing hidden variables in the neuron model, as in neurons with spike-frequency adaptation (SFA). 1a) is more indirect, and learning through gradient descent becomes substantially more difficult. Therefore, the impact of a synaptic weight on the loss function (see Fig. Recurrently connected networks can compute more efficiently because each neuron can participate several times in a network computation, and they are able to solve tasks that require integration of information over time or a non-trivial timing of network outputs according to task demands. Gradients of E are propagated backwards through all layers of the feedforward network to each synapse through a process called backpropagation. In deep learning, this problem is solved for feedforward networks through gradient descent for a loss function E that measures imperfections of current network performance 1. The neuron fires-i.e., emits a spike-when the membrane potential reaches a firing threshold.īut it is an open problem how recurrent networks of spiking neurons (RSNNs) can learn, i.e., how their synaptic weights can be modified by local rules for synaptic plasticity so that the computational performance of the network improves. We consider the arguably most prominent model for spiking neurons in the brain: leaky integrate-and-fire (LIF) neurons, where spikes that arrive from other neurons through synaptic connections are multiplied with the corresponding synaptic weight, and are linearly integrated by a leaky membrane potential. Networks of neurons in the brain differ in at least two essential aspects from deep neural networks in machine learning: they are recurrently connected, forming a giant number of loops, and they communicate via asynchronously emitted stereotypical electrical pulses, called spikes, rather than bits or numbers that are produced in a synchronized manner by each layer of a deep feedforward network.

Prop spike bracelet