Bees, replication, and academic publishing

Bees are insects on which we depend. 35% of food crops grown worldwide require pollinators, but this 35% represents a much greater species diversity than the pollinator-independent crops; their loss would threaten human dietary diversity and consequently health. The most important pollinators are probably bees, many of which are “farmed” for this purpose as much as their honey production.

Bees are under threat from various problems including colony collapse disorder and the Varroa mite. Recently a new threat has emerged, from a fairly new class of pesticide, the neonicotinoids. These compounds were introduced about thirty years ago, and have many advantages; they are far less harmful to humans, fish, and other vertebrates than organophosphorus compounds. However, their effect on bees is a current concern.

In 2017, the results of an observational study were reported by the BBC. I have been unable to find the report of the study, so what follows is from memory. Over 100 sites worldwide were included in the study, which found that in two-thirds of the sites, there were measurable levels of neonic derivatives in honey, and in one-quarter of the sites, these where high enough to be harmful to the bees.

The BBC asked an industry spokesperson to comment. Two comments were reported: first, the levels were too low to pose a danger to human health (no such danger had been suggested in the report); and second, one hundred sites were too few to allow reliable inference about the scale of the problem.

The German chemical company Bayer (which, according to a report I read, is talking about a merger with Monsanto, the company environmentalists love to hate) conducted a large experiment in Germany to test the effects of their poison on bee colonies. They chose two large sites; on one they used neonicotinoids, on the other they didn’t. Then they measured the effect on bee colonies on these sites. An entire issue of the journal Ecotoxicology was devoted to papers resulting from this experiment.

You can probably see the flaw in this already, even if you are not a biologist or a statistician. If a difference is found, you have no way of telling whether it is due to the different treatment (pesticide) or to the different conditions at the two sites.

Indeed, one of the most basic principles of experimental design, one of R. A. Fisher’s “three Rs”, is replication. Statistics provides means of separating these differences, but only if the experiment is replicated, that is, each treatment used on more than one site.

St Andrews biologist Jeremy Greenwood was so annoyed by this that, after some communication with the authors of the study, he, and statistician Rosemary Bailey, wrote a paper pointing out the issue, and submitted it to Ecotoxicology. This is how science works: scientific results must be published, and openly discussed; any flaws will emerge in the discussion.

What happened next throws an interesting light on academic publishing. The editor of the journal was prepared to consider the paper, but downgraded it from a paper to a “Letter to the Editor”. This had two effects. First, open access publication is not available for letters to the editor. Second, the editor offered authors of the original study (one of whom is now a Bayer employee) the right to reply; had a telling answer been available, this could have nipped the criticism in the bud.

Along the way, an environmentalist learned about the paper, and alerted a journalist, who wrote about it. In a fit of pique, the editor accused the authors of breaching confidentiality and threatened to regard the paper as withdrawn. So much for scientific debate. (There was not even the excuse of commercial confidentiality, since the paper simply discussed work already published.) Just imagine if I was barred from telling anyone about my latest theorem until the paper had been accepted by a journal!

In the end the two letters were published; the DOIs are 10.1007/s10646-017-1877-1 and 10.1007/s10646-017-1878-0. If your university has a subscription (Ecotoxicology is a Springer journal) you will be able to take a look, if you want to. Othewise you can read a preprint of the Bailey and Greenwood letter here. My view is that the response doesn’t address the criticism; to me, it reads like “We have a newer and bigger computer, so we can do this sum which involves dividing by zero, which we couldn’t do before”. But I am not an expert.

I see some slight parallel with Alan Sokal’s famous hoax “Transgressing the Boundaries“, published in the journal Social Text twenty years ago. The day the paper was published, he announced that it was a hoax, and garnered a lot of publicity. Sokal described this as an “experiment” with a serious purpose. He was attacking the influence of postmodern notions of relativism applied to science. He had several examples, from Europe, the United States, and India, of actual harm caused by this idea that science is just another discourse no more or less valid than any other. He believed that the progressive Left, rather than supporting this nonsense, should oppose it.

But imagine that an Alan Sokal in a parallel universe had decided that, rather than revealing the hoax, he would proceed to the second phase of the experiment, and leave the paper to lie there until somebody noticed what he had done. This might have taken a while: presumably not many physicists or mathematicians apart from Sokal read Social Text. When eventually some high-school student noticed some of the more obvious errors and wrote to the journal to point them out, would the editor have offered Sokal the right to reply? They may have got more than they had bargained for!