Flagging when the prior distribution is informative

For each parameter (or other qoi), compare the posterior sd to the prior sd. If the posterior sd for any parameter (or qoi) is more than 0.1 times the prior sd, then print out a note: “The prior distribution for this parameter is informative.” Then the user can go back and check that the default prior makes sense for this particular example.

This idea is in our prior choice recommendations wiki, and it’s related to our paper, The prior can often only be understood in the context of the likelihood.

The context is that we’re trying to develop default procedures, which implies default priors and also default diagnostics.

It’s not a bad thing that a prior is informative! It’s just something you’d want to know.

I’ve never actually tried the above idea. I just came up with it one day a few years ago and wrote it down. I like it, though!

P.S. In the top paragraph above, “qoi” stands for “quantity of interest.” Sometimes this is called an “estimand”–that is, something being estimated–but I prefer the more general term “quantity” because in classical frequentist statistics there is a distinction between parameters (fixed but unknown) and predictive quantities (unknown but with probability distributions that in turn depend on fixed parameters). “Quantity of interest” can be any function of data, parameters, latent parameters, missing data, future data and parameters, etc. Also, I associate the the expression “estimand” with the problem formulation in which you have a goal of estimating one particular thing; in contrast, you can have as many quantities of interest as you’d like, and they can be defined after you’ve seen the data.