Bayesian Inference for Big Data with Stochastic Gradient Markov Chain Monte Carlo
This EPSRC project involving Yee Whye Teh (Oxford), Arnaud Doucet (Oxford), and Christophe Andrieu (Bristol) aims to develop both methodologies and theoretical foundations for scalable Markov chain Monte Carlo methods for big data. The starting point was stochastic gradient Langevin dynamics (SGLD) (Welling and Teh 2011), where we have provided theoretical analyses in terms of both asymptotic convergence (Teh et al 2016) as well as weak error expansion (Vollmer et al 2016). We have also developed a range of novel scalable Monte Carlo algorithms based on different techniques.
Publications
2017

L. Hasenclever
,
S. Webb
,
T. Lienart
,
S. Vollmer
,
B. Lakshminarayanan
,
C. Blundell
,
Y. W. Teh
,
Distributed Bayesian Learning with Stochastic Natural Gradient Expectation Propagation and the Posterior Server, Journal of Machine Learning Research, vol. 18, no. 106, 1–37, 2017.
Project: sgmcmc 
X. Lu
,
V. Perrone
,
L. Hasenclever
,
Y. W. Teh
,
S. J. Vollmer
,
Relativistic Monte Carlo, in Artificial Intelligence and Statistics (AISTATS), 2017.
Project: sgmcmc
2016

W. Herlands
,
A. Wilson
,
H. Nickisch
,
S. Flaxman
,
D. Neill
,
W. Van Panhuis
,
E. Xing
,
Scalable Gaussian Processes for Characterizing Multidimensional Change Surfaces, in Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, 2016, 1013–1021.
Project: sgmcmc 
Y. W. Teh
,
A. H. Thiéry
,
S. J. Vollmer
,
Consistency and Fluctuations for Stochastic Gradient Langevin Dynamics, Journal of Machine Learning Research, 2016.
Project: sgmcmc 
S. J. Vollmer
,
K. C. Zygalakis
,
Y. W. Teh
,
Exploration of the (Non)asymptotic Bias and Variance of Stochastic Gradient Langevin Dynamics, Journal of Machine Learning Research (JMLR), 2016.
Project: sgmcmc
2015

T. Lienart
,
Y. W. Teh
,
A. Doucet
,
Expectation Particle Belief Propagation, in Advances in Neural Information Processing Systems (NIPS), 2015.
Project: sgmcmc 
L. Hasenclever
,
S. Webb
,
T. Lienart
,
S. Vollmer
,
B. Lakshminarayanan
,
C. Blundell
,
Y. W. Teh
,
Distributed Bayesian Learning with Stochastic Naturalgradient Expectation Propagation and the Posterior Server, 2015.
Project: sgmcmc 
B. Lakshminarayanan
,
D. M. Roy
,
Y. W. Teh
,
Particle Gibbs for Bayesian Additive Regression Trees, in Proceedings of the International Conference on Artificial Intelligence and Statistics, 2015.
2014

M. Xu
,
B. Lakshminarayanan
,
Y. W. Teh
,
J. Zhu
,
B. Zhang
,
Distributed Bayesian Posterior Sampling via Moment Sharing, in Advances in Neural Information Processing Systems, 2014.
Project: sgmcmc 
B. Paige
,
F. Wood
,
A. Doucet
,
Y. W. Teh
,
Asynchronous Anytime Sequential Monte Carlo, in Advances in Neural Information Processing Systems (NIPS), 2014.
Project: sgmcmc
Software
2016

L. Hasenclever
,
S. Webb
,
T. Lienart
,
S. Vollmer
,
B. Lakshminarayanan
,
C. Blundell
,
Y. W. Teh
,
Posterior Server. 2016.
Project: sgmcmc
2015

B. Lakshminarayanan
,
D. M. Roy
,
Y. W. Teh
,
PGBart. 2015.