Bayesian Inference for Big Data with Stochastic Gradient Markov Chain Monte Carlo

This EPSRC project involving Yee Whye Teh (Oxford), Arnaud Doucet (Oxford), and Christophe Andrieu (Bristol) aims to develop both methodologies and theoretical foundations for scalable Markov chain Monte Carlo methods for big data. The starting point was stochastic gradient Langevin dynamics (SGLD) (Welling and Teh 2011), where we have provided theoretical analyses in terms of both asymptotic convergence (Teh et al 2016) as well as weak error expansion (Vollmer et al 2016). We have also developed a range of novel scalable Monte Carlo algorithms based on different techniques.

Publications

2017

  • X. Lu, V. Perrone, L. Hasenclever, Y. W. Teh, S. J. Vollmer, Relativistic Monte Carlo, in Artificial Intelligence and Statistics (AISTATS), 2017.
    Project: sgmcmc

2016

  • W. Herlands, A. Wilson, H. Nickisch, S. Flaxman, D. Neill, W. Van Panhuis, E. Xing, Scalable Gaussian Processes for Characterizing Multidimensional Change Surfaces, in Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, 2016, 1013–1021.
    Project: sgmcmc
  • Y. W. Teh, A. H. Thiéry, S. J. Vollmer, Consistency and Fluctuations for Stochastic Gradient Langevin Dynamics, Journal of Machine Learning Research, 2016.
    Project: sgmcmc
  • S. J. Vollmer, K. C. Zygalakis, Y. W. Teh, Exploration of the (Non-)asymptotic Bias and Variance of Stochastic Gradient Langevin Dynamics, Journal of Machine Learning Research (JMLR), 2016.
    Project: sgmcmc

2015

  • T. Lienart, Y. W. Teh, A. Doucet, Expectation Particle Belief Propagation, in Advances in Neural Information Processing Systems (NIPS), 2015.
    Project: sgmcmc
  • L. Hasenclever, S. Webb, T. Lienart, S. Vollmer, B. Lakshminarayanan, C. Blundell, Y. W. Teh, Distributed Bayesian Learning with Stochastic Natural-gradient Expectation Propagation and the Posterior Server, 2015.
    Project: sgmcmc
  • B. Lakshminarayanan, D. M. Roy, Y. W. Teh, Particle Gibbs for Bayesian Additive Regression Trees, in Proceedings of the International Conference on Artificial Intelligence and Statistics, 2015.
    Project: bigbayes sgmcmc

2014

  • M. Xu, B. Lakshminarayanan, Y. W. Teh, J. Zhu, B. Zhang, Distributed Bayesian Posterior Sampling via Moment Sharing, in Advances in Neural Information Processing Systems, 2014.
    Project: sgmcmc
  • B. Paige, F. Wood, A. Doucet, Y. W. Teh, Asynchronous Anytime Sequential Monte Carlo, in Advances in Neural Information Processing Systems (NIPS), 2014.
    Project: sgmcmc

Software

2016

  • L. Hasenclever, S. Webb, T. Lienart, S. Vollmer, B. Lakshminarayanan, C. Blundell, Y. W. Teh, Posterior Server. 2016.
    Project: sgmcmc

2014

  • M. Xu, B. Lakshminarayanan, Y. W. Teh, J. Zhu, B. Zhang, SMS: Sampling via Moment Sharing, Advances in Neural Information Processing Systems. 2014.
    Project: sgmcmc