PolySinger: Singing-Voice to Singing-Voice Translation from English to Japanese
arxiv(2024)
Abstract
The speech domain prevails in the spotlight for several natural language
processing (NLP) tasks while the singing domain remains less explored. The
culmination of NLP is the speech-to-speech translation (S2ST) task, referring
to translation and synthesis of human speech. A disparity between S2ST and the
possible adaptation to the singing domain, which we describe as singing-voice
to singing-voice translation (SV2SVT), is becoming prominent as the former is
progressing ever faster, while the latter is at a standstill. Singing-voice
synthesis systems are overcoming the barrier of multi-lingual synthesis,
despite limited attention has been paid to multi-lingual songwriting and song
translation. This paper endeavors to determine what is required for successful
SV2SVT and proposes PolySinger (Polyglot Singer): the first
system for SV2SVT, performing lyrics translation from English to Japanese. A
cascaded approach is proposed to establish a framework with a high degree of
control which can potentially diminish the disparity between SV2SVT and S2ST.
The performance of PolySinger is evaluated by a mean opinion score test with
native Japanese speakers. Results and in-depth discussions with test subjects
suggest a solid foundation for SV2SVT, but several shortcomings must be
overcome, which are discussed for the future of SV2SVT.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined