| Paper ID | MLSP-45.5 |
| Paper Title |
BENIGN OVERFITTING IN BINARY CLASSIFICATION OF GAUSSIAN MIXTURES |
| Authors |
Ke Wang, University of California, Santa Barbara, United States; Christos Thrampoulidis, University of British Columbia, Canada |
| Session | MLSP-45: Performance Bounds |
| Location | Gather.Town |
| Session Time: | Friday, 11 June, 13:00 - 13:45 |
| Presentation Time: | Friday, 11 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Machine Learning for Signal Processing: [MLR-PERF] Bounds on performance |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Deep neural networks generalize well despite being exceedingly overparameterized, but understanding the statistical principles behind this so called benign-overfitting phenomenon is not yet well understood. Recently there has been remarkable progress towards understanding benign-overfitting in simpler models, such as linear regression and, even more recently, linear classification. This paper studies benign-overfitting for data generated from a popular binary Gaussian mixtures model (GMM) and classifiers trained by support-vector machines (SVM). Our approach has two steps. First, we leverage an idea introduced in [Muthukumar et al. 2020] to relate the SVM solution to the least-squares (LS) solution. Second, we derive novel non-asymptotic bounds on the test error of LS solution. Combining the two gives sufficient conditions on the overparameterization ratio and the signal-to-noise ratio that lead to benign overfitting. We corroborate our theoretical findings with numerical simulations. |
