| Paper ID | SPTM-13.3 |
| Paper Title |
WEIGHT IDENTIFICATION THROUGH GLOBAL OPTIMIZATION IN A NEW HYSTERETIC NEURAL NETWORK MODEL |
| Authors |
Elie Leroy, Arthur Marmin, Université Paris-Saclay, CentraleSupélec, Inria, Center for Visual Computing, France; Marc Castella, Samovar, Telecom SudParis, Institut Polytechnique de Paris, France; Laurent Duval, ESIEE Paris, LIGM, Université Gustave-Eiffel and IFP Energies nouvelles, France |
| Session | SPTM-13: Models, Methods and Algorithms 1 |
| Location | Gather.Town |
| Session Time: | Thursday, 10 June, 13:00 - 13:45 |
| Presentation Time: | Thursday, 10 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Signal Processing Theory and Methods: [SSP] Statistical Signal Processing |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Unlike their biological counterparts, simple artificial neural networks are unable to retain information from their past state to influence their behavior. In this contribution, we propose to consider new nonlinear activation functions, whose outputs depend both from the current and past inputs through a hysteresis effect. This hysteresis model is developed in the framework of convolutional neural networks. We then show that, by choosing the nonlinearity in the vast class of rational functions, the identification of the weights amounts to solving a rational optimization problem. For the latter, recent methods are applicable that come with global optimality guarantee, contrary to most optimization methods used in the neural network community. Finally, simulations show that such hysteresis nonlinear activation functions cannot be approximated by traditional ones and illustrate the effectiveness of our weight identification method. |
