Abstract
Cover song generation (CSG) should preserve the melodic and linguistic content of a reference song while recreating the remaining musical components. The state-of-the-art model SongEcho utilizes F0 sequences and voiced/unvoiced (V/UV) tags for conditioning; however, implicit linguistic information from V/UV tags cannot guarantee lyric accuracy, leading to a high phoneme error rate (PER). Inspired by singing voice synthesis (SVS), we propose MPEcho, which integrates a phoneme encoder and a length regulator (LR) into the SongEcho framework. By providing explicit phoneme-level conditioning and precise temporal boundaries, MPEcho significantly reduces PER. To enable this, we developed Phonsa, a Whisper-based automatic transcription model that provides high precision phoneme-level annotations for singing voices, overcoming the scarcity of high-quality audio-phoneme pairs. Experimental results validate the effectiveness of Phonsa for alignment and MPEcho for end-to-end CSG.
Audio Samples Demo
This section shows a gt audio and the generated audio samples by the models reported in Table 4 in the paper. The model ID 1, 2, 3 are trained with M (melody) condition, P (phoneme) condition and M+P conditions respectively. The model ID 9 is the same model with ID 3 but use multi-condition guidance.
| Text Prompt | Lyrics | GT | ID: 1 | ID: 2 | ID: 3 | ID: 9 |
|---|---|---|---|---|---|---|
| slap bass, rhythmic, brass, soulful, upbeat, tight, funk |
從今後就選擇沉默 選擇服從歲月如梭 |
|||||
| bright, happy, pop, acoustic guitar, melodic, radio-friendly, clean vocals |
若非愛的天才 還能不能盼未來 一點愛 就能讓地裂天開 能讓我 把傷感重新掩埋 等你吹散陰霾 活出我的神采 |
|||||
| soul, emotional, keyboard, grooving, romantic, soft, r&b |
她的表情會變的有點怪怪 可是我還是不明白 為什麼她不理不睬 難道要老土的說一聲 對面的女孩看過來 對面的女孩看過來 Won't you just give me a try |
|||||
| nylon guitar, festive, congas, rhythmic, passionate, dance, latin |
我的手你的臉越分越開 記得溫度記得幸福人才能釋懷 如果快樂你忘了帶 想想我們相愛的平凡 那種快樂那種執著 我相信誰都不能取代 |
|||||
| fuzzy guitar, moody, heavy, 90s style, grunge |
我需要一個擁抱 如果你在身邊 就好 我需要一個訊號 是你想念有我的座標 把地圖縮得再小 也不能抵擋難相見的煎熬 我知道我們很好 不會被距離 打倒 |