| Paper ID | MLSP-15.3 |
| Paper Title |
Information and Regularization in Restricted Boltzmann Machines |
| Authors |
Matias Vera, CONICET, Argentina; Leonardo Rey Vega, Universidad de Buenos Aires and CONICET, Argentina; Pablo Piantanida, Université Paris-Saclay, France |
| Session | MLSP-15: Learning Algorithms 2 |
| Location | Gather.Town |
| Session Time: | Wednesday, 09 June, 13:00 - 13:45 |
| Presentation Time: | Wednesday, 09 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Machine Learning for Signal Processing: [MLR-INFO] Information-theoretic learning |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Recent works suggests an interesting interplay between the information flow between inputs features and hidden representations of a learning and the ability of the algorithm to generalize from trained samples to unobserved data. For instance, some the regularization techniques used to control generalization are expected to impact the corresponding information metrics. In this work, we study mutual information in Restricted Boltzmann Machines (RBM) and its relationship with the different regularization techniques. Our results show some evidence on interesting connections between the mutual information (inputs and its representations) with relevant parameters such as: network dimension, matrix norms and dropout probability, which are known to influence the generalization ability of the network. We empirically corroborates our results based on an numerical study. |
