On the Robustness to Misspecification of alpha-Posteriors and Their Variational Approximations

Joint with Marco Avella Medina, José Luis Montiel Olea , and Cynthia Rush

Abstract: $\alpha$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood in posterior calculations and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce the Kullback-Leibler (KL) divergence from the true, but perhaps infeasible, posterior distribution when there is potential parametric model misspecification. To make this point, we derive a Bernstein-von Mises theorem showing convergence in total variation distance of $\alpha$-posteriors and their variational approximations to limiting Gaussian distributions. We use these distributions to evaluate the KL divergence between true and reported posteriors. We show this divergence is minimized by choosing $\alpha$ strictly smaller than one, assuming there is a vanishingly small probability of model misspecification. The optimized value becomes smaller as the the misspecification becomes more severe. The optimized KL divergence increases logarithmically in the degree of misspecification and not linearly as with the usual posterior.

Amilcar Velez
Amilcar Velez
PhD Student in Economics

My research interests include Econometrics, Machine Learning, Experiments, Statistical Decision Theory and Policy Decision.