| Paper ID | SPE-44.4 |
| Paper Title |
IMPROVING RNN TRANSDUCER WITH TARGET SPEAKER EXTRACTION AND NEURAL UNCERTAINTY ESTIMATION |
| Authors |
Jiatong Shi, The Johns Hopkins University, United States; Chunlei Zhang, Chao Weng, Tencent AI Lab, United States; Shinji Watanabe, The Johns Hopkins University, United States; Meng Yu, Dong Yu, Tencent AI Lab, United States |
| Session | SPE-44: Speech Recognition 16: Robust Speech Recognition 2 |
| Location | Gather.Town |
| Session Time: | Thursday, 10 June, 16:30 - 17:15 |
| Presentation Time: | Thursday, 10 June, 16:30 - 17:15 |
| Presentation |
Poster
|
| Topic |
Speech Processing: [SPE-ROBU] Robust Speech Recognition |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Target-speaker speech recognition aims to recognize target-speaker speech from noisy environments with background noise and interfering speakers. This work presents a joint framework that combines time-domain target-speaker speech extraction and Recurrent Neural Network Transducer (RNN-T). To stabilize the joint-training, we propose a multi-stage training strategy that pre-trains and fine-tunes each module in the system before joint-training. Meanwhile, speaker identity and speech enhancement uncertainty measures are proposed to compensate for residual noise and artifacts from the target speech extraction module. Compared to a recognizer fine-tuned with a target speech extraction model, our experiments show that adding the neural uncertainty module significantly reduces 17% relative Character Error Rate (CER) on multi-speaker signals with background noise. The multi-condition experiments indicate that our method can achieve 9% relative performance gain in the noisy condition while maintaining the performance in the clean condition. |
