2021 IEEE International Conference on Acoustics, Speech and Signal Processing
6-11 June 2021 • Toronto, Ontario, Canada
Extracting Knowledge from Information
| Paper ID | IVMSP-8.6 | ||
| Paper Title | NETWORK PRUNING USING LINEAR DEPENDENCY ANALYSIS ON FEATURE MAPS | ||
| Authors | Hao Pan, Zhongdi Chao, Jiang Qian, Bojin Zhuang, Shaojun Wang, Ping An Technology (Shenzhen) Co., Ltd., China; Jing Xiao, Ping An Insurance (Group) Company of China, China | ||
| Session | IVMSP-8: Machine Learning for Image Processing II | ||
| Location | Gather.Town | ||
| Session Time: | Wednesday, 09 June, 13:00 - 13:45 | ||
| Presentation Time: | Wednesday, 09 June, 13:00 - 13:45 | ||
| Presentation | Poster | ||
| Topic | Image, Video, and Multidimensional Signal Processing: [IVCOM] Image & Video Communications | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Network pruning can be achieved by removing redundant channels. In this paper, we regard a channel 'redundant' if its output is linearly dependent with respect to those of other channels. Inspired by this, we propose an efficient pruning method, named as LDFM, by linear dependency analysis on all the feature maps of each individual layer. Specifically, for each layer, by applying the QR decomposition with column pivoting (PQR) on the matrix consisting of all feature maps, those channels corresponding to small absolute diagonal elements of the R matrix from the PQR decomposition are identified as redundant, and are pruned naturally. Although pruning these channels causes loss of information and hence degrades accuracy, the accuracy of the pruned network can be easily recovered by fine-tuning, as the lost information in the pruned channels can be recovered from that in the retained channels. Extensive experiments demonstrate that LDFM makes great improvement on accuracy with similar parameters and FLOPs as other methods, and achieves the state-of-the-art results on several different benchmarks and networks. | ||