Cross-Lingual Singing Voice to Singing Voice Translation: Automatic Translation of Lyrics From A Real Singing Voice to A Synthetic Singing Voice
Author
Term
4. term
Publication year
2024
Submitted on
2024-01-04
Pages
39
Abstract
While cross-lingual synthesis has emerged in Singing Voice Synthesis (SVS), its usage is limited due to language barriers in songwriting. To ease the language barrier, this work presents an attempt at English to Japanese lyrics translation by fine- tuning. This is done jointly with an overarching cascaded solution for singing voice to singing voice trans- lation. The task at hand is to trans- late an English singing voice into a synthesized Japanese voice with lyrics and aligned melody. Such a system is successfully created using state of the art methods, but eval- uations show that the translation model creates poorly singable lyrics. Moreover, a deeper understanding of the Japanese language is needed to create accurate yet singable singing voice to singing voice translations.
While cross-lingual synthesis has emerged in Singing Voice Synthesis (SVS), its usage is limited due to language barriers in songwriting. To ease the language barrier, this work presents an attempt at English to Japanese lyrics translation by fine- tuning. This is done jointly with an overarching cascaded solution for singing voice to singing voice trans- lation. The task at hand is to trans- late an English singing voice into a synthesized Japanese voice with lyrics and aligned melody. Such a system is successfully created using state of the art methods, but eval- uations show that the translation model creates poorly singable lyrics. Moreover, a deeper understanding of the Japanese language is needed to create accurate yet singable singing voice to singing voice translations.
Documents