PAC-Bayes Unleashed: Generalisation Bounds with Unbounded Losses |
| |
Authors: | Maxime Haddouche Benjamin Guedj Omar Rivasplata John Shawe-Taylor |
| |
Institution: | 1.ENS Paris-Saclay, 91190 Gif-sur-Yvette, France;2.Centre for Artificial Intelligence, Department of Computer Science, University College London, London WC1V 6LJ, UK; (O.R.); (J.S.-T.);3.Inria, Lille–Nord Europe Research Centre and Inria London Programme, 59800 Lille, France |
| |
Abstract: | We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval 0;1]). In order to relax this classical assumption, we propose to allow the range of the loss to depend on each predictor. This relaxation is captured by our new notion of HYPothesis-dependent rangE (HYPE). Based on this, we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions. |
| |
Keywords: | statistical learning theory PAC-Bayes generalisation bounds |
|
|