CILVR SEMINAR: Discrete Generative Models for Designing and Sampling Atomistic Data | Practical causal representation learning with an asymmetric prior
In this talk, Nate Gurver will discuss two of his recent projects on applying generative sequence models to atomistic data, in particular protein design and discovery of stable inorganic materials. Taro Makino will talk about Asymmetric Prior Variational Autoencoder (AP-VAE)
Click here to view the recording
Talk1 (Nate Gruver): In this talk I will discuss two of my recent projects on applying generative sequence models to atomistic data, in particular protein design and discovery of stable inorganic materials. Although many people approach these problems by considering symmetries and constraints on continuous objects in 3D space, I will show that relatively small modifications of standard language models can be used to generate atomic structures with desired properties like stability or binding with a target molecule. I will also show how language models can be combined with Bayesian optimization for applications with limited data and expensive validation. Along with introducing new methodology, I will discuss the evaluation in these two papers, including a real-life antibody design campaign with multiple stages of wet-lab synthesis and validation of sampled inorganic materials through extensive density-functional theory (DFT) simulations.
Talk2 (Taro Makino): We often rely on machine learning models to generalize to unseen environments. This problem is called domain generalization, and is extremely challenging due to the presence of environment-specific spurious correlations. We turn to causal representation learning, which aims to learn features that are invariant to the environment. Existing algorithms make unrealistic assumptions, or rely on complex statistical procedures that are too impractical for real-world applications. We propose the Asymmetric Prior Variational Autoencoder to address these weaknesses. In the AP-VAE, there are two latent variables which represent the environment-invariant and environment-specific features. Due to an asymmetry in their priors, these latents can be estimated via standard variational inference, making our approach highly practical. The inferred invariant features are used downstream for out-of-distribution prediction.