Insane Reporting on Acupuncture Trials

Imagine the following:

A drug company promoting a drug for back pain conducts a blinded clinical trial in which patients receive either 1) the drug, 2) a placebo sugar pill that looks the same as the drug or 3) nothing. The results from the trial show that patients who receive both 1 + 2 show more relief from back pain than those receiving no treatment but there is no significant difference between the effect of the placebo pill and the real drug. 

Now imagine a newspaper article reporting on the research chose the headline ‘Scientists find new drug can help to relieve chronic back pain’ followed by a glowing report of how the drug has proven to be effective in treating back pain complete with pleased quotes from the trial authors.

See the problem here? The drug was proven in a clinical trial to be no more effective than an inert sugar pill and yet it is being promoted in the newspaper and by the trial authors as if the trial showed that it is an effective treatment.

This is exactly what happened yesterday when The Times, The Daily Telegraph and The Daily Mirror published extremely complementary articles about a recent trial of acupuncture for chronic back pain.

The trial did find that the acupuncture treatments provided more relief than conventional therapies however it also showed that ‘sham’ acupuncture produced the same benefit. Sham acupuncture is the placebo treatment for this trial, it reproduces the ‘ritual’ and ‘appearance’ of acupuncture treatments but involves no actual needles piercing the skin. Instead a toothpick was used to simulate the sensation of needles piercing the skin. 

The fact that a placebo treatment proved as effective as the real treatment would cause anyone conducting a drug trial to have serious reservations about the efficacy of the drug they were testing. In acupuncture trials however it is becoming increasingly common for sham acupuncture (i.e. the placebo treatment) to be interpreted as if it is just a variation of standard acupuncture!

As a result, when the standard acupuncture treatments are shown in trials to produce results indistinguishable from sham acupuncture treatments the trial is interpreted as being a positive result for acupuncture. This is extremely misleading and it is exactly how the authors of the trial reported on yesterday presented their results.

For anyone truly interested in investigating acupuncture’s effectiveness, the fact that the results from acupuncture treatments are indistinguishable from the results of a placebo treatment should cause them pause to consider is acupuncture really providing anything beyond the placebo effect? Unfortunately, such concerns seem to be increasingly ignored in acupuncture studies which are, by and large, conducted by researchers who have already decided before conducting the trials that acupuncture works. In this particular case the lead researcher for instance commented to the Times that:

We found that simulated acupuncture produced as much benefit as needle acupuncture. That raises questions about how acupuncture works.

The question such a discovery should provoke is if acupuncture works not how it works. The real implications are being replaced with speculations over what such ‘unexpected’ results reveal about the mechanisms by which acupuncture ‘works’. This is not how good research works.

The actual abstract from the study does however present the findings accurately and is worth quoting here in full (emphasis added): 

Although acupuncture was found effective for chronic low back pain, tailoring needling sites to each patient and penetration of the skin appear to be unimportant in eliciting therapeutic benefits. These findings raise questions about acupuncture’s purported mechanisms of action. It remains unclear whether acupuncture or our simulated method of acupuncture provide physiologically important stimulation or represent placebo or nonspecific effects.

Now to me if I was an acupuncturist or a journalist reporting on this, based on the above, I would definitely not regard or promote this as a positive study. The conclusion clearly suggests that it doesn’t matter where you stick the needles and that you apparently do not even need to penetrate the skin to elicit the desired therapeutic effect. All you have to do is pretend to do so and you can get the same positive effect. This suggests that acupuncture treatments ‘work’ via the placebo effect.

This is not how the reports or the authors present the results however and instead we get ‘explanations’ such as the following:

Karen Sherman, also a member of the research team, said that, historically, some types of acupuncture have used non-penetrating needles, possibly explaining the success of the placebo treatment.

That’s certainly one way to try and get around the results but if it’s really a valid explanation you have to wonder why none of the authors and no acupuncturists who commented on the study seem particularly concerned that acupuncturists the world over are unnecessarily inserting needles into people. Essentially, this explanation is akin to the drug company from the analogy explaining the similarity between the drug and the placebo’s effects by saying ‘historically some drugs have included sugar and as such this may possibly explain the success of the placebo treatment’. One would hope most people would recognise the silliness of this justification.

The comments of the lead researcher on the trial, Daniel Cherkin, are also rather telling:

We found that simulated acupuncture produced as much benefit as needle acupuncture. That raises questions about how acupuncture works.

It certainly does raise questions and they are mainly along the lines of what distinguishes acupuncture from an elaborate placebo treatment? This doesn’t seem to be the kind of questions Daniel Cherkin has in mind though…

Overall, I find this to be a rather clear illustration of the positive ‘spin’ that alternative medicine enjoys in the popular media and the rather depressing growing trend for alternative medicine advocates to creatively interpret studies with negative implications to the point were they are presented as actually being positive. Anyone, who looks into the study in detail can see how flawed the reporting on this issue is but most people do not have the time or the inclination to do so. Health reporting in mainstream media is thus misleading the public.

A partial solution to this problem is offered by the growth of independent and freely accesible medical blogs which analyse new studies reported in the media and offer more detailed analysis of what the studies actually show and how reliable they are. Some sites I would particularly recommend are:

1. The NHS ‘Behind the Headlines’ website which examined this study here.
2. Steven Novella’s Neurologica blog which also addressed this study here.
3. The Science Based Medicine blog which features a variety of contributors and includes another more detailed analysis by Steven Novella on the subject.