Probabilistic models provide a powerful framework for modeling diverse data distributions encountered in the real world. These models have found extensive applications in ML-driven knowledge exploration and discovery, including: 1) sequential experiment design and optimization using probabilistic surrogate models like Gaussian processes, and 2) generation of novel designs through generative models. However, a key challenge hindering the application of probabilistic models in real-world scenarios is the lack of scalable solutions that maintain their expressive power. This talk will explore how to overcome this challenge by leveraging the concept of “amortization” through deep neural networks.

First, I will demonstrate how to accelerate the identification of Gaussian process hyperparameters (the major computational bottleneck) by training a single neural network to “amortize” this computationally expensive process across various zero-shot tasks. In the second part, a novel class of generative models will be introduced for flexible and scalable modeling of discrete objects, achieved by learning a neural network to approximate the marginal probability.