Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks
arxiv(2024)
Abstract
Brain-inspired neuromorphic computing with spiking neural networks (SNNs) is
a promising energy-efficient computational approach. However, successfully
training SNNs in a more biologically plausible and
neuromorphic-hardware-friendly way is still challenging. Most recent methods
leverage spatial and temporal backpropagation (BP), not adhering to
neuromorphic properties. Despite the efforts of some online training methods,
tackling spatial credit assignments by alternatives with comparable performance
as spatial BP remains a significant problem. In this work, we propose a novel
method, online pseudo-zeroth-order (OPZO) training. Our method only requires a
single forward propagation with noise injection and direct top-down signals for
spatial credit assignment, avoiding spatial BP's problem of symmetric weights
and separate phases for layer-by-layer forward-backward propagation. OPZO
solves the large variance problem of zeroth-order methods by the
pseudo-zeroth-order formulation and momentum feedback connections, while having
more guarantees than random feedback. Combining online training, OPZO can pave
paths to on-chip SNN training. Experiments on neuromorphic and static datasets
with fully connected and convolutional networks demonstrate the effectiveness
of OPZO with similar performance compared with spatial BP, as well as estimated
low training costs.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined