Learning theory, Optimisation, Monte Carlo methods
I am a DPhil student at the University of Oxford supervised by Patrick Rebeschini and Arnaud Doucet. Before this, I studied Mathematics at Imperial College London. I’m interested in optimisation and theoretical foundations for machine learning. Most recently, I have been interested in measures of generalisation, the use of noise for regularisation and the relationship between optimisation and sampling.
Publications
2021
T. Farghly
,
P. Rebeschini
,
Time-independent Generalization Bounds for SGLD in Non-convex Settings, in Advances in Neural Information Processing Systems 34, 2021.
We establish generalization error bounds for stochastic gradient Langevin dynamics (SGLD) with constant learning rate under the assumptions of dissipativity and smoothness, a setting that has received increased attention in the sampling/optimization literature. Unlike existing bounds for SGLD in non-convex settings, ours are time-independent and decay to zero as the sample size increases. Using the framework of uniform stability, we establish time-independent bounds by exploiting the Wasserstein contraction property of the Langevin diffusion, which also allows us to circumvent the need to bound gradients using Lipschitz-like assumptions. Our analysis also supports variants of SGLD that use different discretization methods, incorporate Euclidean projections, or use non-isotropic noise.
@inproceedings{farghly2021time,
title = {Time-independent Generalization Bounds for SGLD in Non-convex Settings},
author = {Farghly, Tyler and Rebeschini, Patrick},
booktitle = {Advances in Neural Information Processing Systems 34},
year = {2021},
publisher = {Curran Associates, Inc.}
}
