Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time

With recent advances in learning algorithms, recurrent networks of spiking neurons are achieving performance competitive with vanilla recurrent neural networks. Still, these algorithms are limited to small networks of simple spiking neurons and modest-length temporal sequences, as they impose high memory requirements, have difficulty training complex neuron models, and are incompatible with online learning. Here, we show how recently developed ‘Forward-Propagation-Through-Time’ (FPTT) learning combined with novel Liquid Time-Constant spiking neurons resolves these limitations. Applying FPTT to networks of such complex spiking neurons, we demonstrate online learning of exceedingly long sequences while outperforming current online methods and approaching or outperforming offline methods on temporal classification tasks. FPTT’s efficiency and robustness furthermore enables us to directly train a deep and performant spiking neural network for joint object localization and recognition, demonstrating the ability to train large-scale dynamic and complex spiking neural network architectures.

• View at Publisher

Full Text ( Author Manuscript , 2mb )

See Also
techReport Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time B. Yin (Bojian), F. Corradi (Federico) and S.M. Bohte (Sander)
software|data byin-cwi /sFPTT B. Yin (Bojian)