"Binomial Likelihoods and the Polya-Gamma Distribution" (Next Week at the Statistics Seminar)
Three-Toed Sloth 2013-09-25
Summary:
Attention conservation notice: Only of interest if you (1) care about computational statistics, and (2) will be in Pittsburgh next Monday.
Having a talk on Bayesian computational statistics by a Dr. Scott worked so well last time, we're doing it again:
- James Scott, "Binomial Likelihoods and the Polya-Gamma Distribution"
-
Abstract:Bayesian inference for the logistic regression model has
long been recognized as a hard problem. By comparison, Bayesian inference for
the probit model is much easier, owing to the simple latent-variable method of
Albert and Chib (1993) for posterior sampling.
- In the two decades since the work of Albert and Chib on the probit model, there have been many attempts to apply a similar computational strategy to the logit model. These efforts have had mixed results: all such methods are either approximate, or are significantly more complicated than the Albert/Chib method. Perhaps as a result, the Bayesian treatment of the logit model has not seen widespread adoption by non-statisticans in the way that, for example, the Bayesian probit model is used extensively in political science, market research, and psychometrics. The lack of a standard computational approach also makes it more difficult to use the logit link in the kind of rich hierarchical models that have become routine in Bayesian statistics.
- In this talk, I propose a new latent-variable representation for binomial likelihoods. It appeals to a new class of distributions, called the Polya-Gamma family. Although our method involves a different missing-data mechanism from that of Albert and Chib, it is nonetheless a direct analogue of their construction, in that it is both exact and simple. I will describe the Polya-Gamma method in detail; demonstrate its superior efficiency; and highlight a few examples where it has proven helpful. I will conclude by drawing an interesting connection with variational methods.
- Joint work with Jesse Windle and Nicholas Polson.
- Time and place: 4:30--5:30 pm on Monday, 30 September 2013, place TBA (note unusual time)
- In the two decades since the work of Albert and Chib on the probit model, there have been many attempts to apply a similar computational strategy to the logit model. These efforts have had mixed results: all such methods are either approximate, or are significantly more complicated than the Albert/Chib method. Perhaps as a result, the Bayesian treatment of the logit model has not seen widespread adoption by non-statisticans in the way that, for example, the Bayesian probit model is used extensively in political science, market research, and psychometrics. The lack of a standard computational approach also makes it more difficult to use the logit link in the kind of rich hierarchical models that have become routine in Bayesian statistics.