2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information

2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information

Technical Program

Paper Detail

Paper IDSPE-27.6
Paper Title DETECTING ADVERSARIAL ATTACKS ON AUDIOVISUAL SPEECH RECOGNITION
Authors Pingchuan Ma, Petridis Stavros, Maja Pantic, Imperial College London, United Kingdom
SessionSPE-27: Speech Recognition 9: Confidence Measures
LocationGather.Town
Session Time:Wednesday, 09 June, 16:30 - 17:15
Presentation Time:Wednesday, 09 June, 16:30 - 17:15
Presentation Poster
Topic Speech Processing: [SPE-GASR] General Topics in Speech Recognition
IEEE Xplore Open Preview  Click here to view in IEEE Xplore
Virtual Presentation  Click here to watch in the Virtual Conference
Abstract Adversarial attacks pose a threat to deep learning models. However, research on adversarial detection methods, especially in the multi- modal domain, is very limited. In this work, we propose an efficient and straightforward detection method based on the temporal corre- lation between audio and video streams. The main idea is that the correlation between audio and video in adversarial examples will be lower than benign examples due to added adversarial noise. We use the synchronisation confidence score as a proxy for audiovisual correlation and based on it we can detect adversarial attacks. To the best of our knowledge, this is the first work on detection of ad- versarial attacks on audiovisual speech recognition models. We ap- ply recent adversarial attacks on two audiovisual speech recognition models trained on the GRID and LRW datasets. The experimental results demonstrate that the proposed approach is an effective way for detecting such attacks.