I am a DPhil student in Statistics at the University of Oxford, supervised by Prof Yee Whye Teh . I got my Bachelor’s and Master’s degrees in mathematics from Cambridge and worked as a machine learning engineer before joining the department. I have a broad range of interests in statistical machine learning. A large part of my work in Oxford has been on optimal experimental design: how do we design experiments that will be most informative about the process being investigated, whilst minimizing cost? I contribute to the deep probabilistic programming language Pyro and I am the main author of Pyro’s experimental design support. There is a deep mathematical connection between optimal experimental design and mutual information maximization, which is a key tool in unsupervised representation learning. I am currently interested in how we might use mutual information to learn good representations, and how to evaluate the quality of representations once we have them.
Other research interests of mine include the role of equivariance in machine learning, and Bayesian modelling, particularly of network data and transactional data.
Publications
2020
A. Foster
,
M. Jankowiak
,
M. O’Meara
,
Y. W. Teh
,
T. Rainforth
,
A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments, International Conference on Artificial Intelligence and Statistics (AISTATS, to appear), 2020.
We introduce a fully stochastic gradient based approach to Bayesian optimal experimental design (BOED). This is achieved through the use of variational lower bounds on the expected information gain (EIG) of an experiment that can be simultaneously optimized with respect to both the variational and design parameters. This allows the design process to be carried out through a single unified stochastic gradient ascent procedure, in contrast to existing approaches that typically construct an EIG estimator on a pointwise basis, before passing this estimator to a separate optimizer. We show that this, in turn, leads to more efficient BOED schemes and provide a number of a different variational objectives suited to different settings. Furthermore, we show that our gradient-based approaches are able to provide effective design optimization in substantially higher dimensional settings than existing approaches.
@article{foster2020unified,
title = {{A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments}},
author = {Foster, Adam and Jankowiak, Martin and O'Meara, Matthew and Teh, Yee Whye and Rainforth, Tom},
journal = {International Conference on Artificial Intelligence and Statistics (AISTATS, to appear)},
year = {2020}
}
2019
A. Foster
,
M. Jankowiak
,
E. Bingham
,
P. Horsfall
,
Y. W. Teh
,
T. Rainforth
,
N. Goodman
,
Variational Bayesian Optimal Experimental Design, Advances in Neural Information Processing Systems (NeurIPS, spotlight), 2019.
Bayesian optimal experimental design (BOED) is a principled framework
for making efficient use of limited experimental resources. Unfortunately,
its applicability is hampered by the difficulty of obtaining accurate estimates
of the expected information gain (EIG) of an experiment. To address this, we
introduce several classes of fast EIG estimators by building on ideas from
amortized variational inference. We show theoretically and empirically that
these estimators can provide significant gains in speed and accuracy over
previous approaches. We further demonstrate the practicality of our approach
on a number of end-to-end experiments.
@article{foster2019variational,
title = {{Variational Bayesian Optimal Experimental Design}},
author = {Foster, Adam and Jankowiak, Martin and Bingham, Eli and Horsfall, Paul and Teh, Yee Whye and Rainforth, Tom and Goodman, Noah},
journal = {Advances in Neural Information Processing Systems (NeurIPS, spotlight)},
year = {2019}
}
@inproceedings{BloemReddy:etal:2018,
author = {Bloem-Reddy, Benjamin and Foster, Adam and Mathieu, Emile and Teh, Yee Whye},
booktitle = {Conference on Uncertainty in Artificial Intelligence},
title = {Sampling and Inference for Beta Neutral-to-the-Left Models of Sparse Networks},
month = aug,
year = {2018}
}
A. Foster
,
M. Jankowiak
,
E. Bingham
,
Y. W. Teh
,
T. Rainforth
,
N. Goodman
,
Variational Optimal Experiment Design: Efficient Automation of Adaptive Experiments, NeurIPS Workshop on Bayesian Deep Learning, 2018.
Bayesian optimal experimental design (OED) is a principled framework for making efficient use of limited experimental resources. Unfortunately, the applicability of OED is hampered by the difficulty of obtaining accurate estimates of the expected information gain (EIG) for different experimental designs. We introduce a class of fast EIG estimators that leverage amortised variational inference and show that they provide substantial empirical gains over previous approaches. We integrate our approach into a deep probabilistic programming framework, thus making OED accessible to practitioners at large.
@article{foster2018voed,
title = {{Variational Optimal Experiment Design: Efficient Automation of Adaptive Experiments}},
author = {Foster, Adam and Jankowiak, Martin and Bingham, Eli and Teh, Yee Whye and Rainforth, Tom and Goodman, Noah},
journal = {NeurIPS Workshop on Bayesian Deep Learning},
year = {2018}
}
2017
B. Bloem-Reddy
,
E. Mathieu
,
A. Foster
,
T. Rainforth
,
H. Ge
,
M. Lomelí
,
Z. Ghahramani
,
Y. W. Teh
,
Sampling and inference for discrete random probability measures in probabilistic programs, NIPS Workshop on Advances in Approximate Bayesian Inference, 2017.
We consider the problem of sampling a sequence from a discrete random probability measure (RPM) with countable support, under (probabilistic) constraints of finite memory and computation. A canonical example is sampling from the Dirichlet Process, which can be accomplished using its stick-breaking representation and lazy initialization of its atoms. We show that efficiently lazy initialization is possible if and only if a size-biased representation of the discrete RPM is used. For models constructed from such discrete RPMs, we consider the implications for generic particle-based inference methods in probabilistic programming systems. To demonstrate, we implement SMC for Normalized Inverse Gaussian Process mixture models in Turing.
@article{bloemreddy2017rpm,
title = {Sampling and inference for discrete random probability measures in probabilistic programs},
author = {Bloem-Reddy, Benjamin and Mathieu, Emile and Foster, Adam and Rainforth, Tom and Ge, Hong and Lomelí, María and Ghahramani, Zoubin and Teh, Yee Whye},
journal = {NIPS Workshop on Advances in Approximate Bayesian Inference},
year = {2017}
}