| Paper ID | HLT-1.6 |
| Paper Title |
ADAPTABLE MULTI-DOMAIN LANGUAGE MODEL FOR TRANSFORMER ASR |
| Authors |
Taewoo Lee, Min-Joong Lee, Tae Gyoon Kang, Seokyeoung Jung, Minseok Kwon, Yeona Hong, Jungin Lee, Kyoung-Gu Woo, Ho-Gyeong Kim, Jiseung Jeong, Jihyun Lee, Hosik Lee, Young Sang Choi, Samsung Electronics, South Korea |
| Session | HLT-1: Language Modeling 1: Fusion and Training for End-to-End ASR |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 13:00 - 13:45 |
| Presentation Time: | Tuesday, 08 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Human Language Technology: [HLT-LANG] Language Modeling |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
We propose an adapter based multi-domain Transformer based language model (LM) for Transformer ASR. The model consists of a big size common LM and small size adapters. The model can perform multi-domain adaptation with only the small size adapters and its related layers. The proposed model can reuse the full fine-tuned LM which is fine-tuned using all layers of an original model. The proposed LM can be expanded to new domains by adding about 2% of parameters for a first domain and 13% parameters for after second domain. The proposed model is also effective in reducing the model maintenance cost because it is possible to omit the costly and time-consuming common LM pre-training process. Using proposed adapter based approach, we observed that a general LM with adapter can outperform a dedicated music domain LM in terms of word error rate (WER). |
