Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders
ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(2023)
摘要
Diffusion-based generative speech enhancement (SE) has recently received
attention, but reverse diffusion remains time-consuming. One solution is to
initialize the reverse diffusion process with enhanced features estimated by a
predictive SE system. However, the pipeline structure currently does not
consider for a combined use of generative and predictive decoders. The
predictive decoder allows us to use the further complementarity between
predictive and diffusion-based generative SE. In this paper, we propose a
unified system that use jointly generative and predictive decoders across two
levels. The encoder encodes both generative and predictive information at the
shared encoding level. At the decoded feature level, we fuse the two decoded
features by generative and predictive decoders. Specifically, the two SE
modules are fused in the initial and final diffusion steps: the initial fusion
initializes the diffusion process with the predictive SE to improve
convergence, and the final fusion combines the two complementary SE outputs to
enhance SE performance. Experiments conducted on the Voice-Bank dataset
demonstrate that incorporating predictive information leads to faster decoding
and higher PESQ scores compared with other score-based diffusion SE (StoRM and
SGMSE+).
更多查看译文
关键词
speech enhancement,diffusion model,generative model,predictive system
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要