2021 IEEE International Conference on Acoustics, Speech and Signal Processing
6-11 June 2021 • Toronto, Ontario, Canada
Extracting Knowledge from Information
| Paper ID | MLSP-38.6 | ||
| Paper Title | ADVANCES IN MORPHOLOGICAL NEURAL NETWORKS: TRAINING, PRUNING AND ENFORCING SHAPE CONSTRAINTS | ||
| Authors | Nikolaos Dimitriadis, Ecole Polytechnique Fédérale de Lausanne (EPFL), Switzerland; Petros Maragos, National Technical University of Athens (NTUA), Greece | ||
| Session | MLSP-38: Neural Networks for Clustering and Classification | ||
| Location | Gather.Town | ||
| Session Time: | Thursday, 10 June, 16:30 - 17:15 | ||
| Presentation Time: | Thursday, 10 June, 16:30 - 17:15 | ||
| Presentation | Poster | ||
| Topic | Machine Learning for Signal Processing: [MLR-PRCL] Pattern recognition and classification | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | In this paper, we study an emerging class of neural networks, the Morphological Neural networks, from some modern perspectives. Our approach utilizes ideas from tropical geometry and mathematical morphology. First, we state the training of a binary morphological classifier as a Difference-of-Convex optimization problem and extend this method to multiclass tasks. We then focus on general morphological networks trained with gradient descent variants and show, quantitatively via pruning schemes as well as qualitatively, the sparsity of the resulted representations compared to FeedForward networks with ReLU activations as well as the effect the training optimizer has on such compression techniques. Finally, we show how morphological networks can be employed to guarantee monotonicity and present a softened version of a known architecture, based on Maslov Dequantization, which alleviates issues of gradient propagation associated with its "hard" counterparts and moderately improves performance. | ||