sometimes i'm wrong: unearned prestige

"i’ve been known to be critical of eminence and prestige, but it’s really not prestige itself that i’m against. i’m not totally naïve.  one way to think about prestige is just having a good reputation.  i know we’re never going to get rid of reputations, and anyways some research is better than other research, and so deserves a better reputation. the goal shouldn’t be to eliminate that – we want good science to have a good reputation. the problem is unearned prestige. i propose there are at least two wide open paths to unearned prestige, and today i’m going to talk about how transparency can help close those....

but that's not the only way prestige can be unearned.  the other way is when there is a mismatch between a journal's reputation and its actual practices.  if a journal has a reputation for selecting for good science, but what they actually select for is something else, then plenty of research will get a reputation for being good science -- because it's published in that journal -- even when it isn't.  and this can happen even when a close reading of the article actually would have been enough to see that this isn't exemplary science....

so, if the goal is to reduce unearned prestige, how can transparency help? i see a few ways.  first, as i've already talked about, transparency can make it easier to tell the good research from the bad, and to tell the journals that are doing this well from those that are doing it badly, in other words, closing the two loopholes to unearned prestige.  but transparency can also help strengthen the good path - the association between the good research and a good reputation.  the more information we have about what was done, the more calibrated our evaluations can be.  in other words, transparency doesn't guarantee credibility, transparency guarantees* we'll get the credibility we deserve (* void where not paired with scrutiny).  but this is only a possibility that transparency opens up - it doesn't actually guarantee anything.  we have to actually use the transparency to poke and prod at the research to see how solid it is...."