| Paper ID | IVMSP-27.2 |
| Paper Title |
VIOLENCE DETECTION IN VIDEOS BASED ON FUSING VISUAL AND AUDIO INFORMATION |
| Authors |
Wenfeng Pang, Qianhua He, Yongjian Hu, Yanxiong Li, South China University of Technology, China |
| Session | IVMSP-27: Multi-modal Signal Processing |
| Location | Gather.Town |
| Session Time: | Friday, 11 June, 11:30 - 12:15 |
| Presentation Time: | Friday, 11 June, 11:30 - 12:15 |
| Presentation |
Poster
|
| Topic |
Image, Video, and Multidimensional Signal Processing: [IVARS] Image & Video Analysis, Synthesis, and Retrieval |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Determining whether given video frames contain violent content is a basic problem in violence detection. Visual and audio information are useful for detecting violence included in a video, and are usually complementary; however, violence detection studies focusing on fusing visual and audio information are relatively rare. Therefore, we explored methods for fusing visual and audio information. We proposed a neural network containing three modules for fusing multimodal information: 1) attention module for utilizing weighted features to generate effective features based on the mutual guidance between visual and audio information; 2) fusion module for integrating features by fusing visual and audio information based on the bilinear pooling mechanism; and 3) mutual Learning module for enabling the model to learn visual information from another neural network with a different architecture. Experimental results indicated that the proposed neural network outperforms existing state-of-the-art methods on the XD-Violence dataset. |
