| Paper ID | MLSP-9.3 |
| Paper Title |
A Dynamical Systems Perspective on Online Bayesian Nonparametric Estimators with Adaptive Hyperparameters |
| Authors |
Alec Koppel, Amrit Singh Bedi, US Army Research Laboratory, United States; Vikram Krishnamurthy, Cornell University, United States |
| Session | MLSP-9: Learning Theory for Neural Networks |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 16:30 - 17:15 |
| Presentation Time: | Tuesday, 08 June, 16:30 - 17:15 |
| Presentation |
Poster
|
| Topic |
Machine Learning for Signal Processing: [MLR-LEAR] Learning theory and algorithms |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
This paper presents and analyzes constant step size stochastic gradient algorithms in reproducing kernel Hilbert Space (RKHS), which encapsulates various adaptive nonlinear interpolation schemes. The hyperparameters of the function iterates are updated via a distribution that depends on the estimates generated by the algorithm. Using stochastic averaging theory, we show that the estimates generated by the algorithm converge weakly to an algebraically constrained ordinary differential equation. We illustrate this proposed algorithm in an online multi-class classification problem. Specifically, the proposed RKHS-valued stochastic gradient algorithm operating in concert with a Gaussian kernel whose bandwidth stably evolves during training ,performs comparably to when the bandwidth is set according to oracle knowledge of its optimal value. |
