Gene Glass on OA and Meta-Analysis

Having dedicated 35 years of my efforts to meta analysis and 20 to OA, I can’t resist a couple of quick observations: Holding up one set of methods (be they RCT or whatever) as the gold standard is inconsistent with decades of empirical work in meta analysis that shows that “perfect studies” and “less than perfect studies” seldom show important differences in results. If the question at hand concerns experimental intervention, then random assignment to groups may well be inferior as a matching technique to even an ex post facto matching of groups. Randomization is not the royal road to equivalence of groups; it’s the road to probability statements about differences. Claims about the superiority of certain methods are empirical claims. They are not a priori dicta about what evidence can and can not be looked at.