| Paper ID | AUD-2.6 |
| Paper Title |
AN HRNET-BLSTM MODEL WITH TWO-STAGE TRAINING FOR SINGING MELODY EXTRACTION |
| Authors |
Yongwei Gao, Fudan University, China; Xingjian Du, Bilei Zhu, ByteDance AI Lab, China; Xiaoheng Sun, Wei Li, Fudan University, China; Zejun Ma, ByteDance AI Lab, China |
| Session | AUD-2: Audio and Speech Source Separation 2: Music and Singing Voice Separation |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 13:00 - 13:45 |
| Presentation Time: | Tuesday, 08 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Audio and Acoustic Signal Processing: [AUD-MSP] Music Signal Analysis, Processing and Synthesis |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Well-labeled datasets available for melody extraction are scarce, which limits the further advancement of deep learning based methods. To overcome this problem, we propose to use a pitch refinement method to refine the semitone-level pitch sequences decoded from massive melody MIDI files to generate a large number of fundamental frequency (F0) values for model training. Since the refined pitch values used for the first round of training contain errors, a small set of well-labeled data is used for a second round of training. A high-resolution network (HRNet), initially developed for human pose estimation, is introduced for melody extraction. It considers multi-resolution feature learning, making the resulting representation semantically richer. Subsequently, a bidirectional long short-term memory (BLSTM) layer is used to exploit the temporal information of melody. In addition, a new loss function where the unvoiced frames only contribute to voicing detection is also proposed to alleviate the class imbalance problem. Experiment results on three public datasets show that the proposed system outperforms four state-of-the-art algorithms in most cases. |
