| Paper ID | MMSP-2.3 |
| Paper Title |
Teacher-Assisted Mini-Batch Sampling for Blind Distillation using Metric Learning |
| Authors |
Nakamasa Inoue, Tokyo Institute of Technology, Japan |
| Session | MMSP-2: Deep Learning for Multimedia Analysis and Processing |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 14:00 - 14:45 |
| Presentation Time: | Tuesday, 08 June, 14:00 - 14:45 |
| Presentation |
Poster
|
| Topic |
Multimedia Signal Processing: Emerging Areas in Multimedia |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
This paper addresses the problem of blind distillation, which aims to train a student model with unlabeled data under the supervision of a pre-trained teacher model. The proposed framework introduces metric learning to blind distillation. Specifically, teacher-assisted mini-batch (TAM) sampling is proposed, which makes triplets of anchor, positive and negative samples on unlabeled data by using the teacher's knowledge. In addition, we propose a metric-based loss, namely Contrastive Additive Margin (CAM) Softmax loss, which efficiently uses all combinations of triplets on each mini-batch obtained by TAM sampling. In experiments, we show the effectiveness of the proposed framework on face and speaker verification tasks, where student models are trained on unlabeled VoxCeleb videos with a teacher model pre-trained on VGGFace2 images. |
