A master thesis from Aalborg University
Cross-Lingual Singing Voice to Singing Voice Translation: Automatic Translation of Lyrics From A Real Singing Voice to A Synthetic Singing Voice
Author(s)
Term
4. term
Publication year
2024
Submitted on
2024-01-04
Pages
39 pages
Abstract
While cross-lingual synthesis has emerged in Singing Voice Synthesis (SVS), its usage is limited due to language barriers in songwriting. To ease the language barrier, this work presents an attempt at English to Japanese lyrics translation by fine- tuning. This is done jointly with an overarching cascaded solution for singing voice to singing voice trans- lation. The task at hand is to trans- late an English singing voice into a synthesized Japanese voice with lyrics and aligned melody. Such a system is successfully created using state of the art methods, but eval- uations show that the translation model creates poorly singable lyrics. Moreover, a deeper understanding of the Japanese language is needed to create accurate yet singable singing voice to singing voice translations.
While cross-lingual synthesis has emerged in Singing Voice Synthesis (SVS), its usage is limited due to language barriers in songwriting. To ease the language barrier, this work presents an attempt at English to Japanese lyrics translation by fine- tuning. This is done jointly with an overarching cascaded solution for singing voice to singing voice trans- lation. The task at hand is to trans- late an English singing voice into a synthesized Japanese voice with lyrics and aligned melody. Such a system is successfully created using state of the art methods, but eval- uations show that the translation model creates poorly singable lyrics. Moreover, a deeper understanding of the Japanese language is needed to create accurate yet singable singing voice to singing voice translations.
Documents
Colophon: This page is part of the AAU Student Projects portal, which is run by Aalborg University. Here, you can find and download publicly available bachelor's theses and master's projects from across the university dating from 2008 onwards. Student projects from before 2008 are available in printed form at Aalborg University Library.
If you have any questions about AAU Student Projects or the research registration, dissemination and analysis at Aalborg University, please feel free to contact the VBN team. You can also find more information in the AAU Student Projects FAQs.