| Paper ID | SPE-55.2 |
| Paper Title |
JOINT ASR AND LANGUAGE IDENTIFICATION USING RNN-T: AN EFFICIENT APPROACH TO DYNAMIC LANGUAGE SWITCHING |
| Authors |
Surabhi Punjabi, Harish Arsikere, Zeynab Raeesy, Chander Chandak, Nikhil Bhave, Ankish Bansal, Markus Muller, Sergio Murillo, Ariya Rastrow, Andreas Stolcke, Jasha Droppo, Sri Garimella, Roland Maas, Mat Hans, Athanasios Mouchtaris, Siegfried Kunzmann, Amazon, India |
| Session | SPE-55: Language Identification and Low Resource Speech Recognition |
| Location | Gather.Town |
| Session Time: | Friday, 11 June, 14:00 - 14:45 |
| Presentation Time: | Friday, 11 June, 14:00 - 14:45 |
| Presentation |
Poster
|
| Topic |
Speech Processing: [SPE-MULT] Multilingual Recognition and Identification |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Conventional dynamic language switching enables seamless multilingual interactions by running several monolingual ASR systems in parallel and triggering the appropriate downstream components using a standalone language identification (LID) service. Since this solution is neither scalable nor cost- and memory-efficient, especially for on-device applications, we propose end-to-end, streaming, joint ASR-LID architectures based on the recurrent neural network transducer framework. Two key formulations are explored: (1) joint training using a unified output space for ASR and LID vocabularies, and (2) joint training viewed as multi-task optimization. We also evaluate the benefit of using auxiliary language information obtained on-the-fly from an acoustic LID classifier. Experiments with the English-Hindi language pair show that: (a) multi-task architectures perform better overall, and (b) the best joint architecture surpasses monolingual ASR (6.4-9.2% word error rate reduction) and acoustic LID (53.9-56.1% error rate reduction) baselines while reducing the overall memory footprint by up to 46%. |
