LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data - PowerPoint PPT Presentation

LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Brian Trippe , Jonathan Huggins, Raj Agrawal, and Tamara Broderick

LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Brian Trippe , Jonathan Huggins, Raj Agrawal, and Tamara Broderick Cases Controls Diseased Healthy Genomic Study (motivating example) - Goal: Understand relationship between genomic variation & disease outcome - N=20,000 samples — D=500,000 SNPs https://www.ebi.ac.uk/training/

LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Brian Trippe , Jonathan Huggins, Raj Agrawal, and Tamara Broderick Cases Controls Diseased Healthy Genomic Study (motivating example) - Goal: Understand relationship between genomic variation & disease outcome - N=20,000 samples — D=500,000 SNPs https://www.ebi.ac.uk/training/ Generalized Linear Models (GLMs) - Interpretability - E.g. Logistic/Poisson/Negative Binomial Regression

LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Brian Trippe , Jonathan Huggins, Raj Agrawal, and Tamara Broderick Cases Controls Diseased Healthy Genomic Study (motivating example) - Goal: Understand relationship between genomic variation & disease outcome - N=20,000 samples — D=500,000 SNPs https://www.ebi.ac.uk/training/ Generalized Linear Models (GLMs) - Interpretability Bayesian Modeling & Inference - E.g. Logistic/Poisson/Negative - Coherent uncertainty quantification Binomial Regression Problem: Super-linear scaling with D

LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Brian Trippe , Jonathan Huggins, Raj Agrawal, and Tamara Broderick Cases Controls Diseased Healthy Genomic Study (motivating example) - Goal: Understand relationship between genomic variation & disease outcome - N=20,000 samples — D=500,000 SNPs https://www.ebi.ac.uk/training/ Generalized Linear Models (GLMs) - Interpretability Bayesian Modeling & Inference - E.g. Logistic/Poisson/Negative - Coherent uncertainty quantification Binomial Regression Problem: Super-linear scaling with D

LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Brian Trippe , Jonathan Huggins, Raj Agrawal, and Tamara Broderick Cases Controls Diseased Healthy Genomic Study (motivating example) - Goal: Understand relationship between genomic variation & disease outcome - N=20,000 samples — D=500,000 SNPs https://www.ebi.ac.uk/training/ Generalized Linear Models (GLMs) - Interpretability Bayesian Modeling & Inference - E.g. Logistic/Poisson/Negative - Coherent uncertainty quantification Binomial Regression Problem: Super-linear scaling with D We present LR-GLM , a method with linear scaling in D and theoretical guarantees on quality

How does it work?

How does it work? Cartoon Example - Logistic Regression with two correlated features

How does it work? Cartoon Example - Logistic Regression with two correlated features

How does it work? Cartoon Example - Logistic Regression with two correlated features Uncertainty in Effect Sizes

How does it work? Cartoon Example - Logistic Regression with two correlated features Uncertainty in Effect Sizes n o i t a m r o f n I f o s L t i o t t L l e I n f o r m a t i o n

How does it work? Cartoon Example - Logistic Regression with two correlated features Uncertainty in Effect Sizes n o The LR-GLM Approximation i t a m r o We ignore the least informative f n I | f o directions { s L t i o t i β ) ≈ p ( y i | x i UU T β ) t L l p ( y i | x T e I n f o r m a t i o n

How does it work? Cartoon Example - Logistic Regression with two correlated features Uncertainty in Effect Sizes n o The LR-GLM Approximation i t a m r o We ignore the least informative f n I | f o directions { s L t i o t i β ) ≈ p ( y i | x i UU T β ) t L l p ( y i | x T e I n f o r m a t i o n

How does it work? Cartoon Example - Logistic Regression with two correlated features Uncertainty in Effect Sizes n o The LR-GLM Approximation i t a m r o We ignore the least informative f n I | f o directions { s L t i o t i β ) ≈ p ( y i | x i UU T β ) t L l p ( y i | x T e I n f o r m a t i Approximation Quality o n - Exact when data are low rank - We prove: Approximation is close when the data are approximately low rank

How does it work? Cartoon Example - Logistic Regression with two correlated features Uncertainty in Effect Sizes n o The LR-GLM Approximation i t a m r o We ignore the least informative f n I | f o directions { s L t i o t i β ) ≈ p ( y i | x i UU T β ) t L l p ( y i | x T e I n f o r m a t i Approximation Quality o n - Exact when data are low rank - We prove: Approximation is close when the data are approximately low rank

Does it Work?

Does it Work? Evaluate by comparing exact means and uncertainties (slow) against our approximation (fast)

Does it Work? Evaluate by comparing exact means and uncertainties (slow) against our approximation (fast) Post. Uncertainty Estimation Post. Mean Estimation Approx. Uncertainty Approx. Mean Exact Mean Exact Uncertainty

Does it Work? Evaluate by comparing exact means and uncertainties (slow) against our approximation (fast) Post. Uncertainty Estimation Post. Mean Estimation Approx. Uncertainty Approx. Mean Exact Mean Exact Uncertainty We rigorously show… - Rank of approximation defines a computational-statistical trade-o ff - The approximation is conservative (overestimates uncertainty) - For high-dimensional, correlated data, LR-GLM closely approximates the exact posterior up to 5X faster!

Does it Work? Evaluate by comparing exact means and uncertainties (slow) against our approximation (fast) Post. Uncertainty Estimation Post. Mean Estimation Approx. Uncertainty Approx. Mean Exact Mean Exact Uncertainty We rigorously show… - Rank of approximation defines a computational-statistical trade-o ff - The approximation is conservative (overestimates uncertainty) - For high-dimensional, correlated data, LR-GLM closely approximates the exact posterior up to 5X faster!

Does it Work? Evaluate by comparing exact means and uncertainties (slow) against our approximation (fast) Post. Uncertainty Estimation Post. Mean Estimation Approx. Uncertainty Approx. Mean Exact Mean Exact Uncertainty We rigorously show… - Rank of approximation defines a computational-statistical trade-o ff - The approximation is conservative (overestimates uncertainty) - For high-dimensional, correlated data, LR-GLM closely approximates the exact posterior up to 5X faster!

Does it Work? Evaluate by comparing exact means and uncertainties (slow) against our approximation (fast) Post. Uncertainty Estimation Post. Mean Estimation Approx. Uncertainty Approx. Mean Exact Mean Exact Uncertainty We rigorously show… - Rank of approximation defines a computational-statistical trade-o ff - The approximation is conservative (overestimates uncertainty) - For high-dimensional, correlated data, LR-GLM closely approximates the exact posterior up to 5X faster!

Does it Work? LR-GLM: High-Dimensional Bayesian Inference Using Low-Rank Data Approximations Evaluate by comparing exact means and uncertainties (slow) against Brian L. Trippe , Jonathan H. Huggins, Raj Agrawal and Tamara Broderick our approximation (fast) Paper: proceedings.mlr.press/v97/trippe19a Post. Uncertainty Estimation Post. Mean Estimation Poster: Pacific Ballroom #214 Approx. Uncertainty Approx. Mean Exact Mean Exact Uncertainty We rigorously show… - Rank of approximation defines a computational-statistical trade-o ff - The approximation is conservative (overestimates uncertainty) - For high-dimensional, correlated data, LR-GLM closely approximates the exact posterior up to 5X faster!