Recently the Guardian ran an article by David Colquhoun, a professor of pharmacology at University College London, complaining about peer review. His complaints were innocuous; what was interesting was his example. How bad is peer review? he said. Look what gets published! He pointed to a study of the efficacy of acupuncture and included graphs of the results. “It’s obvious at a glance that acupuncture has at best a tiny and erratic effect on any of the outcomes that were measured,” he wrote.
Except it wasn’t. There were four graphs. Each had two lines — one labelled “acupuncture,” the other labelled “control”. You might think to assess the effect of acupuncture you compare the two lines. That wasn’t true. The labels were misleading. The “acupuncture” group got acupuncture early in the experiment; the “control” group got acupuncture late in the experiment. Better names would have been early treatment and late treatment. You could not allow for this “at a glance”. It was too complicated. With this design, if acupuncture were effective the difference between the two lines should be “erratic”.
The paper’s data analysis is poor. To judge the efficacy of acupuncture, their main comparison used only the data from the first 26 weeks. They could have used data from all 52 weeks. That is, they ignored half of their data when trying to answer their main question. Colquhoun could have criticized that, but he didn’t.
Colquhoun’s criticism was so harsh and shallow, apparently he is biased against acupuncture. But there are two big things few pharmacology professors appear to know. One is how to stimulate the immune system. This should be central in pharmacology, but it isn’t. Half of why I think fermented foods are so important is that I think they stimulate the immune system. (The other half is they improve digestion.) There are plenty of less common ways to do this. The phenomenon of hormesis suggests that small doses of all sorts of poisons, including radiation, stimulate repair systems. The evidence behind the hygiene hypothesis suggests that dirt improves the immune systems of children. Bee stings have been used to treat arthritis. And so on. In this context, sticking needles into someone, which puts a small amount of bacteria into their blood, is not absurd. Acupuncture also allowed patients to share their symptoms, the value of which Jon Cousins has emphasized.
The other big thing Colquhoun doesn’t seem to know is the absurdity of the chemical imbalance theory of depression. Speaking of ridiculous, that’s ridiculous. Which plays a larger role in modern medicine — antidepressants or acupuncture? If you want criticize peer review, criticize the chemical imbalance theory. It is as if peer reviewers have been saying, yes, the earth really is flat for fifty years. Perhaps this is ending. During a talk that Robert Whitaker gave at the Massachusetts General Hospital in January, he was told by doctors there that the chemical-imbalance theory was an “outdated model”.
Thanks to Dave Lull and Gary Wolf.