Coded Speech Enhancement Using Neural Network-Based Vector-Quantized Residual Features.

Various approaches have been proposed to improve the quality of the speech coded at low bitrates. Recently, deep neural networks have also been used for speech coding, providing a high quality of speech with low bitrates. Although designing an entire codec with neural networks may be more effective, backward compatibility with the existing codecs can be desirable so that the systems with the legacy codec can still decode the coded bitstream. In this paper, we propose to generate side information based on neural networks for an existing codec and enhance the decoded speech with another neural networks using the side information. The vector-quantization variational autoencoder (VQ-VAE) is applied to generate vector-quantized side information and reconstruct the residual features, which are the difference between the features extracted from the original and decoded signals. The post-processor in the decoder side, which is another neural network, takes the decoded signal of the main codec and the reconstructed residual features to estimate the features for the original signal. Experimental results show that the proposed method can significantly improve the quality of the enhanced signals with additional bitrate of 0.6 kbps for two of the implementations of the high-efficiency advanced audio coding (HE-AAC) v1.