Anthony Caterini

• GitHub
• Google Scholar

High-Dimensional Statistics, Monte Carlo Methods, Variational Inference

I am a DPhil student in Statistics at the University of Oxford, supervised by Profs. Arnaud Doucet and Dino Sejdinovic. Generally, I am interested in statistical methods for inference in high-dimensional data, and I am currently working on using Hamiltonian methods in Variational Inference.

Publications

2021

• A. Caterini , R. Cornish , D. Sejdinovic , A. Doucet , Variational Inference with Continuously-Indexed Normalizing Flows, in Uncertainty in Artificial Intelligence (UAI), 2021.
  
  Continuously-indexed flows (CIFs) have recently achieved improvements over baseline normalizing flows on a variety of density estimation tasks. CIFs do not possess a closed-form marginal density, and so, unlike standard flows, cannot be plugged in directly to a variational inference (VI) scheme in order to produce a more expressive family of approximate posteriors. However, we show here how CIFs can be used as part of an auxiliary VI scheme to formulate and train expressive posterior approximations in a natural way. We exploit the conditional independence structure of multi-layer CIFs to build the required auxiliary inference models, which we show empirically yield low-variance estimators of the model evidence. We then demonstrate the advantages of CIFs over baseline flows in VI problems when the posterior distribution of interest possesses a complicated topology, obtaining improved results in both the Bayesian inference and maximum likelihood settings.

  @inproceedings{CatCorSejDou2021,
    author = {Caterini, Anthony and Cornish, Rob and Sejdinovic, Dino and Doucet, Arnaud},
    title = {{{Variational Inference with Continuously-Indexed Normalizing Flows}}},
    booktitle = {Uncertainty in Artificial Intelligence (UAI)},
    year = {2021}
  }
  

2020

• R. Cornish , A. Caterini , G. Deligiannidis , A. Doucet , Relaxing bijectivity constraints with continuously indexed normalising flows, in ICML, 2020, 2133–2143.
  

  @inproceedings{cornish2020relaxing,
    title = {Relaxing bijectivity constraints with continuously indexed normalising flows},
    author = {Cornish, R. and Caterini, A. and Deligiannidis, G. and Doucet, A.},
    booktitle = {ICML},
    pages = {2133--2143},
    year = {2020},
    organization = {PMLR}
  }
  

2019

• D. Watson-Parris , S. Sutherland , M. Christensen , A. Caterini , D. Sejdinovic , P. Stier , Detecting Anthropogenic Cloud Perturbations with Deep Learning, in ICML 2019 Workshop on Climate Change: How Can AI Help?, 2019.
  
  One of the most pressing questions in climate science is that of the effect of anthropogenic aerosol on the Earth’s energy balance. Aerosols provide the ‘seeds’ on which cloud droplets form, and changes in the amount of aerosol available to a cloud can change its brightness and other physical properties such as optical thickness and spatial extent. Clouds play a critical role in moderating global temperatures and small perturbations can lead to significant amounts of cooling or warming. Uncertainty in this effect is so large it is not currently known if it is negligible, or provides a large enough cooling to largely negate present-day warming by CO2. This work uses deep convolutional neural networks to look for two particular perturbations in clouds due to anthropogenic aerosol and assess their properties and prevalence, providing valuable insights into their climatic effects.

  @inproceedings{Watetal2019,
    author = {Watson-Parris, Duncan and Sutherland, Sam and Christensen, Matthew and Caterini, Anthony and Sejdinovic, Dino and Stier, Philip},
    title = {{Detecting Anthropogenic Cloud Perturbations with Deep Learning}},
    booktitle = {ICML 2019 Workshop on Climate Change: How Can AI Help?},
    year = {2019}
  }
  

2018

• A. Caterini , A. Doucet , D. Sejdinovic , Hamiltonian Variational Auto-Encoder, in Advances in Neural Information Processing Systems (NeurIPS), 2018, to appear.
  
  Variational Auto-Encoders (VAEs) have become very popular techniques to perform inference and learning in latent variable models as they allow us to leverage the rich representational power of neural networks to obtain flexible approximations of the posterior of latent variables as well as tight evidence lower bounds (ELBOs). Combined with stochastic variational inference, this provides a methodology scaling to large datasets. However, for this methodology to be practically efficient, it is necessary to obtain low-variance unbiased estimators of the ELBO and its gradients with respect to the parameters of interest. While the use of Markov chain Monte Carlo (MCMC) techniques such as Hamiltonian Monte Carlo (HMC) has been previously suggested to achieve this, the proposed methods require specifying reverse kernels which have a large impact on performance. Additionally, the resulting unbiased estimator of the ELBO for most MCMC kernels is typically not amenable to the reparameterization trick. We show here how to optimally select reverse kernels in this setting and, by building upon Hamiltonian Importance Sampling (HIS), we obtain a scheme that provides low-variance unbiased estimators of the ELBO and its gradients using the reparameterization trick. This allows us to develop a Hamiltonian Variational Auto-Encoder (HVAE). This method can be reinterpreted as a target-informed normalizing flow which, within our context, only requires a few evaluations of the gradient of the sampled likelihood and trivial Jacobian calculations at each iteration.

  @inproceedings{CatDouSej2018,
    author = {Caterini, A.L. and Doucet, A. and Sejdinovic, D.},
    title = {{{Hamiltonian Variational Auto-Encoder}}},
    booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
    pages = {to appear},
    year = {2018}
  }
  

• A. Caterini , D. E. Chang , Deep Neural Networks in a Mathematical Framework. Springer, 2018.
  

  @book{caterini2018deep,
    title = {Deep Neural Networks in a Mathematical Framework},
    author = {Caterini, A. and Chang, D. E.},
    year = {2018},
    publisher = {Springer}
  }
  

2017

• A. Caterini , A Novel Mathematical Framework for the Analysis of Neural Networks, Master's thesis, University of Waterloo, 2017.
  

  @mastersthesis{caterini2017novel,
    title = {{A Novel Mathematical Framework for the Analysis of Neural Networks}},
    author = {Caterini, A.},
    year = {2017},
    school = {University of Waterloo}
  }
  

2016

• A. Caterini , D. E. Chang , A Geometric Framework for Convolutional Neural Networks, ArXiv e-prints:1608.04374, 2016.
  
  @unpublished{caterini2016geometric,
    title = {A Geometric Framework for Convolutional Neural Networks},
    author = {Caterini, A. and Chang, D. E.},
    journal = {ArXiv e-prints:1608.04374},
    year = {2016}
  }
  

2015

• M. Winlaw , M. Hynes , A. Caterini , H. De Sterck , Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark, in Parallel and Distributed Systems (ICPADS), 2015 IEEE 21st International Conference on, 2015, 682–691.
  

  @inproceedings{winlaw2015algorithmic,
    title = {{Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark}},
    author = {Winlaw, M. and Hynes, M. and Caterini, A. and De Sterck, H.},
    booktitle = {Parallel and Distributed Systems (ICPADS), 2015 IEEE 21st International Conference on},
    pages = {682--691},
    year = {2015},
    organization = {IEEE}
  }