Open Access articles not cited more? Really? On Davis, Methodology & Advocacy

Many studies have reported a statistically significant increase in citations for articles whose authors make them OA by self-archiving them. To show that this citation advantage is not causal but just a self-selection artifact (because authors selectively self-archive their better, more citeable papers), you first have to replicate the advantage for the self-archived OA articles in your sample, and then show that the advantage is absent for the articles made OA at random. But Davis showed only that the citation advantage was absent altogether in his sample. The likely reason is that the sample was much too small (36 journals, 712 articles randomly OA, 65 self-archived OA, 2533 non-OA). In a recent study (Gargouri et al 2010) we controlled for self-selection with mandated (obligatory) OA rather than random OA. The far larger sample (1984 journals, 3055 articles mandatorily OA, 3664 self-archived OA, 20,982 non-OA) revealed a statistically significant citation advantage of about the same size for both self-selected and mandated OA. If and when Davis's requisite self-selected self-archiving control is ever tested, the outcome will either be (1) the usual significant OA citation advantage in the self-archiving control condition that most other published studies have reported -- in which case the absence of the citation advantage in Davis's randomized condition would indeed be evidence that the citation advantage had been a self-selection artifact that was then successfully eliminated by the randomization -- or (more likely, I should think) (2) there will be no significant citation advantage in the self-archiving control condition either, in which case the Davis study will prove to have been just a non-replication of the usual significant OA citation advantage (perhaps because of Davis's small sample size, the fields, or the fact that most of the non-OA articles become OA on the journal's website after a year).