Sleep-Dependent Memory Replay Enables Brain-Like Robustness in Neural Networks.

Sleep is known to play a crucial role in memory and learning processes. During sleep, the replay of neural activity patterns associated with recent memories is believed to contribute to memory consolidation. Inspired by the sleep replay process in the biological system, we proposed a brain-like memory replay algorithm consisting of three phases. First, an artificial neural network (ANN) underwent learning, mimicking the cognitive processes of the awake stage. Next, we conducted a simulation where slow oscillation sleep or awake stages were simulated in a spiking neural network (SNN). The network connections and initial synaptic weights are identical to those of the trained ANN. Lastly, the performance of the ANN is evaluated and validated during the awake phase using the synaptic weights converted from the SNN. We applied this brain-inspired memory replay algorithm to a two-digit MNIST classification task. We found that as the level of noise increases, the model exhibits a significant improvement in performance after a period of sleep compared to the awake state. These findings provide compelling evidence that sleep-dependent memory consolidation can significantly enhance the network performance and improve its the robustness. Overall, this study sheds light on the potential advantages of incorporating sleep-like processes into neural networks, offering valuable insights into the field.