eLife press release: Deterministic thinking led to a nonsensical statement

Quoting from a report produced by the biology journal eLife, Dorothy Bishop writes:

The eLife report states:

It is important to note that we don’t ascribe value to the decision to review. Our aim is to produce high-quality reviews that will be of significant value but we are not able to review everything that is submitted.

That is hard to believe: if you really were just ignoring quality considerations, then you should decide on which papers to review by lottery. I think this claim is not only disingenuous but also wrong-headed. If you have a limited resource – reviewer capacity – then you should be focusing it on the highest quality work. But that judgement should be made on the basis of research question and design, and not on results.

I agree with Bishop regarding journal reviewing—even better, in my opinion, would be post-publication review, but let’s also do better with the institutions we already have.

The question then remains, how could the authors of the report have written that they don’t ascribe value to the decision to review, given that they screen papers in order to decide whether to review them?

My guess is that the authors of the report fell victim to a sort of deterministic thinking. A more accurate statement, I think, would be something like, “We ascribe value to the decision to review, but these are noisy decisions. Often we end up reviewing papers that turn out to be unsuitable for the journal, and we’re sure that often we decline to review papers that would have been just fine for the journal.” But it’s hard for them to say that, as this would imply uncertainty and error. From a deterministic perspective, the only two possible statements are: (a) We don’t ascribe value to the decision to review, or (b) All the papers we decline to review are unsuitable for the journal. They didn’t want to say (b), so they said (a).

P.S. The above should not be taken as any criticism of eLife’s policies or of its report (other than that one statement of theirs, “It is important to note that we don’t ascribe value to the decision to review”).

P.P.S. It is possible to improve this sort of thing! Remember a few years ago when PNAS updated its slogan from the false “PNAS publishes only the highest quality scientific research” to the more accurate “PNAS strives to publish only the highest quality scientific research.”