Anthony Caterini

• GitHub
• Google Scholar

I am a DPhil student in Statistics at the University of Oxford, supervised by Profs. Arnaud Doucet and Dino Sejdinovic. Generally, I am interested in statistical methods for inference in high-dimensional data, and I am currently working on using Hamiltonian methods in Variational Inference.

Publications

2018

• A. Caterini , A. Doucet , D. Sejdinovic , Hamiltonian Variational Auto-Encoder, in Advances in Neural Information Processing Systems (NeurIPS), 2018, to appear.
  
  Abstract BibTeX

  arXiv

  Video

  Variational Auto-Encoders (VAEs) have become very popular techniques to perform inference and learning in latent variable models as they allow us to leverage the rich representational power of neural networks to obtain flexible approximations of the posterior of latent variables as well as tight evidence lower bounds (ELBOs). Combined with stochastic variational inference, this provides a methodology scaling to large datasets. However, for this methodology to be practically efficient, it is necessary to obtain low-variance unbiased estimators of the ELBO and its gradients with respect to the parameters of interest. While the use of Markov chain Monte Carlo (MCMC) techniques such as Hamiltonian Monte Carlo (HMC) has been previously suggested to achieve this, the proposed methods require specifying reverse kernels which have a large impact on performance. Additionally, the resulting unbiased estimator of the ELBO for most MCMC kernels is typically not amenable to the reparameterization trick. We show here how to optimally select reverse kernels in this setting and, by building upon Hamiltonian Importance Sampling (HIS), we obtain a scheme that provides low-variance unbiased estimators of the ELBO and its gradients using the reparameterization trick. This allows us to develop a Hamiltonian Variational Auto-Encoder (HVAE). This method can be reinterpreted as a target-informed normalizing flow which, within our context, only requires a few evaluations of the gradient of the sampled likelihood and trivial Jacobian calculations at each iteration.

  @inproceedings{CatDouSej2018,
    author = {Caterini, A.L. and Doucet, A. and Sejdinovic, D.},
    title = {{{Hamiltonian Variational Auto-Encoder}}},
    booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
    pages = {to appear},
    year = {2018}
  }
  

• A. Caterini , D. E. Chang , Deep Neural Networks in a Mathematical Framework. Springer, 2018.
  
  BibTeX

  @book{caterini2018deep,
    title = {Deep Neural Networks in a Mathematical Framework},
    author = {Caterini, A. and Chang, D. E.},
    year = {2018},
    publisher = {Springer}
  }
  

2017

• A. Caterini , A Novel Mathematical Framework for the Analysis of Neural Networks, Master's thesis, University of Waterloo, 2017.
  
  BibTeX

  @mastersthesis{caterini2017novel,
    title = {{A Novel Mathematical Framework for the Analysis of Neural Networks}},
    author = {Caterini, A.},
    year = {2017},
    school = {University of Waterloo}
  }
  

2016

• A. Caterini , D. E. Chang , A Geometric Framework for Convolutional Neural Networks, ArXiv e-prints:1608.04374, 2016.
  
  BibTeX

  arXiv

  @unpublished{caterini2016geometric,
    title = {A Geometric Framework for Convolutional Neural Networks},
    author = {Caterini, A. and Chang, D. E.},
    journal = {ArXiv e-prints:1608.04374},
    year = {2016}
  }
  

2015

• M. Winlaw , M. Hynes , A. Caterini , H. De Sterck , Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark, in Parallel and Distributed Systems (ICPADS), 2015 IEEE 21st International Conference on, 2015, 682–691.
  
  BibTeX

  @inproceedings{winlaw2015algorithmic,
    title = {{Algorithmic Acceleration of Parallel ALS for Collaborative Filtering: Speeding up Distributed Big Data Recommendation in Spark}},
    author = {Winlaw, M. and Hynes, M. and Caterini, A. and De Sterck, H.},
    booktitle = {Parallel and Distributed Systems (ICPADS), 2015 IEEE 21st International Conference on},
    pages = {682--691},
    year = {2015},
    organization = {IEEE}
  }