NICE Guidelines

Shortcomings in trial design can result in the “wrong” outcome: An example based on acupuncture

~ complementaryandalternative ~ Leave a comment

I have recently written about the problems with the decision by the National Institute for Health and Care Excellence (NICE) to include acupuncture in their draft guidelines on chronic pain. Here, I’m going to look at one of the studies they used as a basis for that decision in more detail. It includes a number of serious issues that means the results are unlikely to be accurate.

Study Overview

The study in question is “German Randomized Acupuncture Trial for chronic shoulder pain (GRASP) – a pragmatic, controlled, patient-blinded, multi-centre trial in an outpatient care environment”. The hypothesis for the study was “There is a significant difference for the responder rate in the verum group compared to that of sham and of COT. The patients with a pain reduction of at least 50% compared to baseline were classified as responders.” (Note that “verum” in this case refers to the acupuncture treatment group and COT is short for conservative orthopaedic treatment). Patient self-assessment of pain levels on a visual analogue scale (VAS) was used as the measurement. This primary endpoint was assessed 3 months after the completion of the 6 weeks of treatment.

The study claimed: “The trial indicates that Chinese acupuncture is an effective alternative to conventional orthopaedic treatment for CSP”. Due to the many flaws in this study, this trial doesn’t actually show this at all.

Main Flaws in the Study

Here are some of the main issues with this study:

  • Use of traditional acupuncture points. The study states that the following points were used: “ventral – Lung 1, 2; ventrolateral – Large Intestine 4, 11, 14, 15; lateral – Sanjiao 5, 13, 14; dorsal – Small Intestine 3, 9”. Traditional Chinese acupuncture is based on Qi and meridians, which are ideas that conflict with modern science. These traditional acupuncture points don’t actually exist.
  • Delay in publication. Patients were recruited between 1997 and 1999 but the paper was only submitted for publication in March 2009, a full ten years later. There may have been valid reasons for this but it is certainly unusual and raises concerns. Even if there were genuine reasons for such an extended delay, it seems reasonable to expect that the paper authors would struggle to remember full details of the trial after such an extended period.
  • Drop-outs and how these were handled in the analysis. Drop-outs were high, particularly in the sham arm. Only 74 out of 135 (just 55%) in the sham arm reached the primary endpoint. In the treatment arm 128 out of 154 (83%) reached this point, along with 106 out of 135 (79%) in the conservative orthopaedic treatment (COT) arm. In all cases the drop-outs were “treated as non responders”. The problem with this is the very high drop-out rate in the sham arm. Treating the people who have dropped out from the sham arm as “non responders” is likely to underestimate the improvement in this arm. Because the drop-out rate in the treatment arm was much lower this will make the treatment appear more effective than the sham. It is a mistake from the authors to not consider this issue in their analysis. They don’t even mention it. One possible solution they could have used is to assume that drop-outs in the sham arm are “responders” (https://www.sciencedirect.com/science/article/pii/B9780128042175000084 ). This is the most cautious way of assessing the results but is a way of minimising the impact of the very high drop-out in the sham arm.
  • Adverse events. Adverse event reporting is very important for any trial of a treatment. This is what the authors say: “Although we did not observe any serious adverse events (SAEs), it cannot be excluded that this is due to an underreporting by centres not being experienced in reporting SAEs.” Were the centres not given a clear protocol for how and when to report adverse events? Inadequate reporting of SAEs raises safety concerns about the trial.
  • Contextual effects and blinding. The sham acupuncture points used in this trial were located on the leg. Yet, this was a treatment for shoulder pain. The authors provide this explanation “the further away sham points were located from verum points, the greater the observed difference between verum and sham treatment”. I would suggest that there is a very obvious explanation for this. When sham points are chosen that are nowhere near the area of the body needing treatment, this makes the sham less believable. It seems likely that many patients will realise that they are receiving the sham treatment when they need treatment for their shoulder and acupuncture needles are placed in their leg! This means the patients are effectively unblinded at this point, resulting in less contextual effects and invalidating the comparison between treatment and sham.

Professor Edzard Ernst also raised concerns about this paper after it was published, particularly focusing on the potential for the observed benefits to be down to contextual effects rather than the treatment itself.

Conclusion

There are many serious issues with this trial including with the treatment approach, inadequate blinding, analysis issues and safety concerns. The results from this study therefore shouldn’t really be used to inform treatment choices and it definitely isn’t evidence of the effectiveness of acupuncture. It’s disappointing that NICE have used such poor quality studies as a basis for their recommendations in their draft guideline on chronic pain.

Why have NICE included acupuncture in their guidelines for chronic pain?

~ complementaryandalternative ~ 3 Comments

Earlier this month the National Institute for Health and Care Excellence (NICE) released a draft of their new guideline on chronic pain and one of the treatment recommendations was acupuncture. Although acupuncture is commonly used to treat pain, recent research found that the evidence is conflicting and inconclusive. NICE usually takes a rigorous approach to developing guidelines, so how did they end up recommending a treatment that does not have good evidence of effectiveness? This is a complex issue that deserves further investigation.

Summary of conclusions:

1. The evidence in favour of acupuncture for chronic pain is weak and in many cases conflicting.

2. Traditional Chinese acupuncture is based on entirely false beliefs in qi, meridians and other pre-scientific concepts. It shouldn’t be included in any health guidelines.

3. NICE seem to have made a mistake in recommending acupuncture in their draft guidelines for chronic pain. Hopefully they will correct this mistake in the final guidelines.

What is acupuncture?

There are many different types of acupuncture, with all of them being based in some way on placing needles in various parts of the body. There are two main ideas that underpin acupuncture: (i) Traditional Chinese acupuncture based on qi (or chi) and meridians and (ii) Modern acupuncture based on a neurophysiological model (that acupuncture needles stimulate nerve endings and alter brain function). Qi and meridians are based on pre-scientific beliefs and should have no place in modern healthcare.

What do the NICE guidelines for chronic pain say about acupuncture?

The NICE guidelines state:

Consider a course of acupuncture or dry needling, within a traditional Chinese or Western acupuncture system, for people aged 16 years and over to manage chronic primary pain, but only if the course:

  • is delivered in a community setting, and
  • is delivered by a band 7 (or lower) healthcare professional, and
  • is made up of no more than 5 hours of healthcare professional time (the number and length of sessions can be adapted within these boundaries).

Besides the recommendation for acupuncture itself, it’s concerning that NICE refer to “a traditional Chinese or Western acupuncture system” (emphasis mine). As previously stated, traditional Chinese acupuncture is based on Qi and meridians. These are concepts that conflict with modern science and have never been shown to actually exist.

Looking in the more detailed Evidence review for acupuncture, things get even worse. In that section, NICE refer directly to the concept of Qi and talk about this as if it’s a real thing. Here is what they say:

“Therapists trained in traditional Chinese medicine operate under the assumption of optimising the flow of the vital energy “Qi” in the body.”

Now there is no such thing as “vital energy” but, for some reason that I’m struggling to understand, NICE appears not to know this. They also go on to say:

“protagonists of traditional Chinese medicine (TCM) choose distant points in their attempts to harmonise the perceived misbalance of body functions and emotions.”

This is complete nonsense, yet here is the organisation that is responsible for producing evidence-based health and social care guidance in the UK taking about it in their guideline for chronic pain.

I have written previously on this blog about other guidelines produced by NICE such as this post about acupuncture for low back pain. Their approach to developing guidelines is usually extremely rigorous. I have also personally worked with them on other topics and seen the same thorough approach there too. I am staggered, and I don’t use this word lightly, that an organisation that is usually so careful and thorough is referring to traditional Chinese acupuncture, Qi and related concepts in the evidence used to develop their guideline on chronic pain.

What does the actual evidence say?

Within NICE’s Evidence review for acupuncture they include a series of “Clinical evidence statements”. For acupuncture versus sham acupuncture they have a large number of these statements. In almost all cases the evidence is labelled as low or very low quality. Is evidence of such low quality sufficient to make a recommendation in favour of a treatment? I would argue that it is not. When you take into account that several of the papers used to prepare these evidence statements are based on traditional Chinese acupuncture, I would suggest that the argument in favour of acupuncture looks even more dubious.

I have looked more closely at some of the papers that NICE refer to in their Evidence review for acupuncture. I have focused on those papers that a) compare acupuncture to sham and b) have a reasonably large sample size. A detailed explanation of why I have done this is beyond the scope of this article but in short: a) comparison with sham is important to control for non-specific treatment effects such as placebo and b) studies with larger sample sizes are more likely to provide accurate results than those with smaller sample sizes. Overall the quality of papers is poor with numerous methodological flaws in most of them, ranging from problems with blinding to issues with the statistical analysis. Even where these poor quality papers found a benefit the improvement was actually pretty small and unlikely to be clinically significant. Importantly, several of the studies are also based on the use of traditional Chinese acupuncture. Here is an example quote from one of the papers:

“The most frequently diagnosed TCM syndromes were Liver Qi stagnation (44.4%), followed by Spleen and Kidney Yang deficiency (21%) and Yin deficiency (13.6%).”

None of these things is recognised or accepted as a medical diagnosis and they are based on an outdated and incorrect approach to carrying out such a diagnosis. Taking all of this into account, it is hard to understand why NICE think acupuncture is a viable treatment for chronic pain.

Why did NICE include acupuncture as a recommendation for chronic pain?

The evidence for acupuncture as a treatment for chronic pain is pretty thin on the ground and in a recent review has been described as conflicting and inconclusive. The traditional Chinese medicine approach to acupuncture is based on things that do not exist and makes no logical sense. Why then did NICE include it as a recommendation in their guidelines for chronic pain? I had given this significant thought and not come up with a good answer. I did wonder if they were just short of interventions to include as a treatment for chronic pain and had concluded that acupuncture is the best of a bad bunch. However, I think that on balance there are two better explanations:

  • As described in the post from Steve Novella on Science Based Medicine, NICE have failed to consider Science Based Medicine altogether. This is a serious failing for an organisation that has such an important role in creating guidelines for health and social care.
  • One of the members of the NICE committee is Jens Foell, a GP who makes use of acupuncture in his practice and has even authored papers supporting acupuncture. In the Declaration of Interests register for the committee members he declares “I have been a member of the council of BMAS, the British Medical Acupuncture Society and member of the editorial board of “Acupuncture in Medicine” in the 12 months preceding joining the advisory I stood down from these posts in the last weeks”. This is someone who is clearly a strong supporter of acupuncture and provides a potential explanation for the inclusion of acupuncture within the guideline. The DoI was handled by having him “Declare and withdraw from drafting recommendations on acupuncture.” However, it also states that “The committee member will remain in the room for presentation of the evidence and may be asked to respond to specific questions from the committee regarding this topic.” Is having a fellow member of the committee who could speak confidently and strongly in favour of acupuncture likely to influence the committee’s decision? Absolutely! My thanks to David Colquhoun for highlighting this issue.

The other important point raised in the Science Based Medicine post is the risk of legitimising acupuncture and the potential for acupuncturists to use this NICE recommendation to market acupuncture for other conditions. Quoting from the SBM article “By recommending acupuncture for pain, based upon very weak evidence of a clinically tiny effect in the face of a lack of a plausible mechanism, you are throwing patients to the wolves.”

To conclude, it is a serious concern that NICE has failed to apply any sort of science-based approach when selecting treatments to include in their guideline. This has resulted in them recommending a treatment based on Qi, meridians and other nonsensical concepts. Even putting the lack of plausible scientific basis to one side, the evidence supporting acupuncture as a treatment for chronic pain is very weak. I am not the only one to have these concerns. Besides the article by Steve Novella on SBM, Edzard Ernst has also written about this and reaches similar conclusions. I think that NICE have made a serious mistake here and hope that they correct this error when they produce the final version of the guideline.

Further reading – other reviews of the guidelines

Last updated 12/04/21

Should you see a chiropractor or an osteopath for your migraines?

~ complementaryandalternative ~ 2 Comments

migraineI’ve previously written about the problems with chiropractors and osteopaths claiming to be able to treat all types of headaches. The Advertising Standards Authority (ASA) permits chiropractors and osteopaths to advertise their treatments for only two specific types of headache:

  • Headache arising from the neck (cervicogenic)
  • Migraine prevention

In this post I’m going to look more closely at chiropractic and osteopathy treatment for the prevention and treatment of migraines. How strong is the evidence behind these treatments?

Summary of conclusions:

1. Based on the currently available evidence there doesn’t seem to be any good reason to recommend osteopathy or chiropractic for the prevention or treatment of migraines.

2. The treatments recommended in the NICE guidelines for prevention and treatment of migraines are all available from a GP rather than a chiropractor or osteopath.

3. Further research in this area may be worthwhile as there is currently relatively little good research.

Evidence for chiropractic and osteopathy for migraine prevention

I don’t know the source of the ASA guideline permitting chiropractors and osteopaths to advertise their treatments for migraine prevention. However, it’s probable that at least part of the source is the Effectiveness of manual therapies: the UK evidence report (Bronfort report). Within this publication it states: “The impetus for this report stems from the media debate in the United Kingdom (UK) surrounding the scope of chiropractic care and claims regarding its effectiveness particularly for non-musculoskeletal conditions.” That’s not a very good reason for carrying out research. Regardless, let’s look at what it says about migraine prevention.

The Bronfort report states that there is “Moderate quality evidence that spinal manipulation has an effectiveness similar to a first-line prophylactic prescription medication (amitriptyline) for the prophylactic treatment of migraine”. This is based on 2 systematic reviews of manual therapy for migraine headache. The Bronfort report chooses to discard the results of 1 of the systematic reviews because it “evaluated study quality using a scale that is no longer recommended by the Cochrane Collaboration and did not apply evidence rules for their conclusions”. They chose instead to follow only the conclusions from the other systematic review. Who was the lead author for that systematic review? Gert Bronfort, the same person who is lead author for the Bronfort report. There is clearly a potential conflict of interest here so let’s have a look at the actual papers that formed the basis of both systematic reviews (there are only 3 of them and the same papers are in both reviews):

The first thing that surprises me here is that both systematic reviews found only 3 trials related to manual therapy and migraines. This suggests that it’s an area that is under-researched and makes drawing strong conclusions difficult. Furthermore, the conclusion drawn by the Bronfort report that “Moderate quality evidence that spinal manipulation has an effectiveness similar to a first-line prophylactic prescription medication (amitriptyline) for the prophylactic treatment of migraine” is rather strange. It is based on just a single trial. Drawing conclusions from a single trial is always problematic and we would usually like to see similar results independently replicated in another trial before making any significant recommendations. In fact, given the apparent success of SMT found in this trial, which was conducted over 20 years ago, it’s surprising that there isn’t further published research looking at the same thing in the 12 year period to 2010 (when the Bronfort report was published).

There has been some additional research since 2010. Here is a trial published in 2017: Chiropractic spinal manipulative therapy for migraine: a three-armed, single blinded, placebo, randomized controlled trial . In this trial there were three groups: 1) active – Chiropractic spinal manipulative therapy (CSMT), 2) placebo – sham treatment, 3) control – normal pharmacological management. The authors concluded that “the effect of CSMT observed in our study is probably due to a placebo response”. In other words, the treatment was not effective. The other conclusion drawn by the authors is interesting: “It is possible to conduct a manual-therapy RCT with concealed placebo”. Some osteopaths and chiropractors claim that it’s not possible to adequately test their treatments in a randomised controlled trial (RCT). However, here we have an RCT where the authors concluded that this was entirely possible. It would therefore seem that it’s no longer reasonable to use the excuse that it’s not possible to test osteopathy or chiropractic treatments in an RCT.

NICE Guidelines

For a more definitive and recent view it makes sense to look at the NICE guidelines. There are two key publications from NICE on this topic: NICE guidelines on Headaches in over 12s: diagnosis and management and the Clinical Knowledge Summary for Migraine. Neither of those publications makes any recommendation in favour of the use of the main treatments provided by chiropractors or osteopaths (manual therapy). If you look into the evidence behind the guidelines in more detail it’s easy to see why. This is what the NICE GDG (Guideline Development Group) had to say: “For migraine, there was one study showing some benefit. The GDG were concerned that the evidence reviewed was of low to very low quality with a lot of uncertainty in the effect estimates, and that rare adverse events may be severe when they do occur. It was agreed that better evidence was required to make a recommendation.”

The NICE guidelines recommend the use of medication-based treatments for the prevention and treatment of migraines. This should be in addition to use of a headache diary as well as providing appropriate information and support. All of these treatments and approaches are available from a GP.

Conclusion

Overall, the lack of research in this area makes it difficult to draw any significant conclusions about the effectiveness of osteopathy or chiropractic for migraines. There is certainly not enough evidence in order to make a recommendation in favour of using these treatments. It makes sense to use the NICE guidelines as the most reliable source of evidence and they recommend treatments that are available from a GP rather than those from a chiropractor or osteopath.  I would summarise the current position as follows:

Based on the currently available evidence there doesn’t seem to be any good reason to recommend osteopathy or chiropractic for the treatment or prevention of migraines.

It would, however, make sense to conduct further research to see whether osteopathy or chiropractic could be of benefit. (The NICE guidelines make a similar recommendation for further research.)

In view of the lack of evidence, it’s surprising that the ASA guidelines allow chiropractors and osteopaths to advertise their treatments for migraines. The ASA guidelines would probably benefit from a review and update based on the evidence that’s available, in particular the NICE guidelines. In the meantime, if you have migraines it is best that you receive evidence-based treatments and these are all available from your GP.

 

Last updated 06/08/19

A recent paper suggests Spinal Manipulation Therapy (SMT) is as effective as recommended treatments for low back pain. Chiropractors celebrate. Should they?

~ complementaryandalternative ~ Leave a comment

Introduction

Spinal Manipulation Therapy (SMT) is the primary treatment used by most chiropractors and osteopaths as well as being used by some physiotherapists. This study set out to compare Spinal Manipulation Therapy (SMT) with recommended therapies, non-recommended therapies, sham SMT and SMT as adjuvant therapy. It concluded that SMT showed similar benefits to “recommended interventions”. When it was published it was met with trumpet fanfares by a number of chiropractic organisations, as well as others, and claimed as providing strong justification for their treatments. In actual fact it does not do this so let’s explore further.

Summary of conclusions:

1. There are currently no really effective treatments for low back pain.

2. At best, SMT results in a modest average clinical effect. It has also not been shown to be more effective than sham SMT.

3. It’s not currently possible to know whether SMT is safe.

4. It is not currently clear whether SMT is cost effective or not.

5. Taking into account the limited clinical effectiveness and unclear position on safety and cost effectiveness it doesn’t seem to make sense to recommend SMT as a first-line treatment.

Background

Back pain is a widespread problem and SMT is currently listed as a second-line or adjunctive treatment in a number of national guidelines, such as the UK NICE Guidelines as well as the Lancet low back pain guidelines from last year. It’s not recommended as a first-line treatment because current evidence shows it is not sufficiently effective when used alone.

The study and its conclusions

The study was a systematic review and meta-analysis that compared SMT with recommended therapies, non-recommended therapies, sham SMT and SMT as adjuvant therapy. It identified recommended and non-recommended interventions based on guidelines from US, UK and The Netherlands. It’s not clear why these particular guidelines were picked. The Lancet series used a similar set but with Denmark in place of The Netherlands. Perhaps the inclusion of the guideline from The Netherlands is due to the lead author being based there? Whilst it’s understandable that someone will be more familiar and comfortable with guidelines from the country in which they live and work this does introduce a degree of bias into the results and it would be good to see that acknowledged in the paper.

The study concluded that SMT showed similar benefits to “recommended interventions”. It also concluded that SMT vs “non-recommended interventions” shows a statistically but not clinically significant benefit for pain but a statistically and clinically significant benefit for “back specific functional status”. There have been some criticisms, but more on that shortly.

The reaction

A number of chiropractic organisations, as well as others, latched onto this study and claimed it is strong justification for their treatments. Take for instance, this tweet from the British Chiropractic Association:

BCA on Twitter_ _As part of the package of care, chiropractors often _ - twitter.com

It suggests that SMT, the treatment most favoured by most chiropractors, should be considered a first-line treatment for chronic low back pain.

There is also this from the Chiropractic Research Council which includes the comment “supersedes other reviews and publications e.g The Lancet Series which only recommended SMT as a second line treatment option. Groundbreaking stuff!”

As I’ll explain shortly, this review is not particularly “groundbreaking” and the justification for SMT being a first-line treatment is nowhere near as strong as these comments would suggest.

Criticisms of the paper

A thorough and clear critique of the paper was submitted by Mary O’Keeffe and Neil O’Connell. This highlighted a number of issues, including:

  1. SMT was not shown to be better than sham SMT. As this is the only reliable way of assessing efficacy of SMT, it suggests that SMT is not effective as a treatment.
  2. The choice of “recommended” and “non-recommended” therapies was strange and does not align with guidelines. For instance, treatment with paracetamol (which is not recommended by clinical guidelines) was included as “recommended” therapy.
  3. The authors do not seem to interpret their results in line with their own thresholds for determining clinical effectiveness.

The authors submitted a response in which they stand by their original conclusions. They say that SMT cannot be adequately blinded and that therefore trials that compare with other treatments are more valuable than those comparing SMT to sham. In response to point 2 above about choice of “recommended” and “non-recommended” therapies, they say they have undertaken additional analysis taking account of this point and have still arrived at similar results. As far as I can tell, they haven’t responded to the point about thresholds for clinical effectiveness.

Neil O’Connell has sent some further replies on twitter about comparison with sham and the fact that we might see the same minimal benefit for an ineffective treatment:

Neil O'Connell on Twitter_ _Thanks for a considered and thoughtful re_ - twitter.com

It seems likely that this discussion will continue. Hopefully the authors will continue to engage as it’s good to have a healthy debate on complex issues like this. What’s clear is that SMT has not been shown to be particularly effective for low back pain and this new study doesn’t change that.

Safety of SMT

The paper says “About half of the studies examined adverse events (table 2). In most of these studies it was unclear how and whether adverse events were registered systematically; therefore, these data might be unreliable and not accurate for incidence.” In other words: due to a lack of systematic reporting of adverse events, it’s impossible to adequately assess the safety of SMT. The paper does go on to say “clinicians should ensure that patients are fully informed of potential risks before treatment”. This is good advice but I wonder how much this actually happens in practice? Do the main people who carry out SMT (osteopaths, chiropractors and some physiotherapists) ensure that their patients are fully informed of potential risks before they start treatment? Or do they just go ahead and start treatment without doing this?

Cost effectiveness

The paper states “it remains to be determined whether SMT is a cost effective option for the treatment of chronic low back pain.” As more research is needed to assess the cost effectiveness of SMT it would make sense to conduct this research before deciding whether to recommend it as a first-line treatment. If it turns out to be significantly more expensive than other similarly effective treatments then it would not make sense to recommend it as a first-line treatment.

Placing these results in the context of other research

In the section of the paper “Comparison with other studies” it is stated that “our results are consistent with other recently published high quality systematic reviews and guidelines that recommend SMT”. I disagree that it is consistent with guidelines, e.g. from NICE, because they have SMT as a second-line treatment option whereas this paper suggests it is as good as currently recommended first-line treatments.

Future research

The paper says “Given the considerable data available, we can now calculate within reasonable certainty the effect of SMT in this setting as well as the impact of a future, methodologically well conducted trial (as determined by the prediction intervals).” In other words, don’t keep doing more similar research in this area because it isn’t going to add anything. This is good advice and is something that researchers should conclude much more often than currently happens. Far too often a study that doesn’t show effectiveness suggests that “more research is needed”. This is a waste of time and money as there is no point in continuing to carry out research when it isn’t going to add anything new.

Conclusions

Overall, this is a decent piece of research and shows fairly similar results to other previous research. It concludes “The evidence suggests that SMT results in a modest, average clinical effect at best”. What’s most strange is not the paper itself but the hype that has surrounded it. There were many “trumpet fanfares” from chiropractic organisations suggesting that this vindicates their treatments and should be used to promote them. It doesn’t actually do this. It shows that SMT isn’t a particularly effective treatment, that it’s not possible to adequately assess its safety and it is not known whether it’s cost effective or not. This is hardly a justification for shouting from the rooftops about how good it is.

What this study does do is add to the body of evidence that indicates that, unfortunately, there remains no good treatment for low back pain. This presents an issue for the many people who suffer with low back pain as there are no really effective treatments. When presented with a number of treatments that are not particularly effective, what is someone with back pain supposed to do? There is no easy answer. However, exercise is known to provide general health benefits and the NHS provides activity guidelines. The importance of these general health benefits should not be understated and is something that SMT cannot provide. In addition, the NICE guidelines recommend exercise as the first-line treatment as well as highlighting the importance of continuing with normal activities as far as possible (self-management). These approaches have the advantage of empowering the patient whereas SMT is a purely passive treatment that does not encourage self-management. I can understand the temptation for someone with back pain to want to go and “get fixed” by seeing a chiropractor, osteopath or physiotherapist who uses SMT. Unfortunately, the evidence indicates that it is not particularly effective and it’s not clear how safe it is. Bearing in mind that it doesn’t provide other general health benefits or empower the patient and is not currently considered a first-line treatment by national guidelines it should probably remain a second-line or adjunctive treatment.

Should you have acupuncture for your low back pain? A review of the Lancet series and 3 national guidelines

~ complementaryandalternative ~ 1 Comment

acupunctureIn March 2018 The Lancet published a series on low back pain consisting of three papers and a related podcast. This is an important topic due to the large number of people who suffer with back pain at some point during their lives and it’s good that the Lancet have given it particular focus. One thing that’s particularly interesting in this series is that acupuncture is recommended as a “second-line or adjunctive treatment option”. The UK NICE guidelines for low back pain and sciatica, published in November 2016, made it clear that acupuncture is no longer a recommended treatment in the UK for low back pain with or without sciatica. So why does this Lancet series recommend it as a second-line or adjunctive treatment option? This is an interesting question that is worthy of further investigation.

Summary of conclusions:

1. Based on the available evidence, there is no good reason to recommend acupuncture as a treatment for either acute or chronic low back pain with or without sciatica.

2. It is impossible to say whether acupuncture is safe or not.

3. 2 out of the 3 national guidelines used for the Lancet paper do not recommend acupuncture and there are reasons to question the validity of the recommendation in favour of acupuncture from the US guideline.

The Lancet paper that covers this is effectively a systematic review of 3 national guidelines on low back pain from Denmark, the UK and the US. Of the 3 guidelines, the only one that recommends acupuncture is the one from the US. Those from the UK and Denmark do not recommend acupuncture. In their press release to accompany the latest version of the UK guidelines, NICE clearly states “Acupuncture for treating low back pain is not recommended because evidence shows it is not better than sham treatment”. Usually when developing guidelines, all relevant evidence is reviewed internationally. This leaves two key open questions:

  1. Why did the US guidelines reach a different conclusion to those from the UK and Denmark?
  2. Why did the Lancet series recommend acupuncture as a “second-line or adjunctive treatment option” when only 1 in 3 of the national guidelines they reviewed recommended it?

Differences between UK and US guidelines

There are a number of differences between the approach taken by the UK and US guideline developers.

Firstly, the UK guidelines were quite focused on the need to show effects over and above contextual or placebo effects. This is what they say (highlighting mine):

“The GDG first discussed the necessity of a body of evidence to show specific intervention effects, that is, over and above any contextual or placebo effects. It was therefore agreed that if placebo-controlled evidence (or sham acupuncture) is available, this should inform decision making in preference to contextual effects, but that the effect sizes compared with usual care would be important to consider if effectiveness relative to placebo, or sham, has been demonstrated.”

This is different to the approach taken by the US guideline which has made recommendations based on the difference between acupuncture and no acupuncture without clear evidence of a specific effect over and above placebo. This is an important distinction because in order to show that a treatment is actually effective it’s important to show a benefit over and above the placebo effect.

When assessing the evidence for acupuncture compared with placebo / sham, this is what the UK GDG (Guideline Development Group) has to say:

“For the placebo/sham controlled evidence in the low back pain population, the GDG agreed that no clinical benefit was seen for pain or function.”

The other key difference between the UK and US guidelines is the approach to analysing the available evidence. The US guideline used systematic reviews directly without investigating the individual studies that made up the review. In contrast, the UK guideline used systematic reviews as a way to identify individual RCTs (Randomised Controlled Trials) but did not include the systematic reviews themselves. The UK approach is clearly significantly more work than the US approach. However, it is likely to result in a more accurate assessment of the evidence base for the following reasons:

  1. The systematic reviews may have set out to answer a slightly different question to the one that the guideline developers are seeking to answer.
  2. It enables the guideline developers to undertake a consistent assessment of all of the primary studies. (It’s unlikely that the assessment of studies will be consistent across multiple systematic reviews.)
  3. It removes the risk of a systematic review reaching inaccurate conclusions based on poor quality primary studies. (Unfortunately, some systematic reviews do report conclusions that are actually based on poor quality or highly biased primary RCTs.)

I’m not trying to suggest that the conclusions reached by the US guideline are wrong. In science it is not normal to talk in such strong terms as right and wrong but instead to talk about the probability of something being correct or what the best available evidence indicates. However, it is likely that the UK guideline is closer to an accurate answer than the US guideline for the reasons highlighted above. The UK guideline clearly states that acupuncture should not be recommended as a treatment for people with low back pain with or without sciatica.

Acupuncture recommendations in the US guidelines

Here are the 2 recommendations for the use of acupuncture in the US guidelines:

  • For acute or sub-acute low back pain, acupuncture is recommended as one of several treatments with the quality of evidence stated as low.
  • For chronic low back pain, acupuncture is recommended as one of several treatments with the quality of evidence stated as moderate.

Let’s take a closer look at the evidence that has informed these recommendations for acupuncture.

Acute or Subacute Low Back Pain

The following statement is made in the guideline:

“Low-quality evidence showed that acupuncture resulted in a small decrease in pain intensity compared with sham acupuncture with nonpenetrating needles, but there were no clear effects on function (76–78). Low-quality evidence showed that acupuncture slightly increased the likelihood of overall improvement compared with NSAIDs (76, 79–83).”

It is difficult to assess this in more detail as for several of the papers used only the abstract is readily available. However, low-quality evidence of a small decrease in pain intensity is hardly compelling evidence on which to base a recommendation for the use of acupuncture.

Chronic Low Back Pain

The following statement is made in the guideline:

“Low-quality evidence showed that acupuncture was associated with moderate improvement in pain relief immediately after treatment and up to 12 weeks later compared with sham acupuncture, but there was no improvement in function (125–130). Moderate-quality evidence showed that acupuncture was associated with moderately lower pain intensity and improved function compared with no acupuncture at the end of treatment (125). Low-quality evidence showed a small improvement in pain relief and function compared with medications (NSAIDs, muscle relaxants, or analgesics) (125).”

It’s interesting that the only item with moderate-quality evidence is a comparison between acupuncture and no acupuncture. In this situation there is no control for the placebo effect and the observed improvement may actually be entirely down to the placebo effect. What’s more important is the difference between acupuncture and sham acupuncture and there the evidence is low quality. It’s therefore surprising that the guideline authors have concluded in their summary recommendations that there is moderate-quality evidence for the use of acupuncture. The more detailed information that they present here clearly indicates that there is actually only low-quality evidence.

Risk of Harm

The US guideline states that “Low-quality evidence showed no reported harms or serious adverse events associated with … acupuncture”. However, adverse event reporting is unfortunately omitted from many trials of complementary and alternative medicine including acupuncture. This is a serious ethical issue that makes it difficult to objectively assess the risk associated with acupuncture and other CAM therapies. Edzard Ernst frequently highlights this on his blog, with an example relating to acupuncture here. All of this leads us to this conclusion: It is impossible to say whether acupuncture is safe as adverse events are not adequately reported or investigated.

Recommendations from the Lancet paper

The Lancet paper is effectively a systematic review of 3 national guidelines. Only the US guidelines recommend acupuncture as a treatment for low back pain. The guidelines from the UK and Denmark say that it isn’t recommended. Why then does the Lancet paper make a recommendation for the use of acupuncture as a second-line or adjunctive treatment? It was not clear to me from the paper why this had happened, but I was able to discuss it on Twitter with one of the authors and he explained that the intent was to reflect what the guidelines recommended. There was lack of consensus between the guidelines but the US one recommended acupuncture and therefore the paper authors included it as an adjunctive treatment.

It is important to note that there are several other good recommendations from this paper and it’s also clear that a great deal of work must have gone into preparing this. Despite my concerns about the acupuncture recommendations, the authors should be commended for a valuable contribution to the management of low back pain.

Conclusion

Based on the available evidence, there is no good reason to recommend acupuncture as a treatment for either acute or chronic low back pain with or without sciatica. Whilst there are unfortunately no treatments that have been shown to be highly effective there are other interventions, such as exercise and self-management, that have a stronger indication of effectiveness than acupuncture. Patients should therefore be guided towards these treatments. In addition, future research should focus on improving and optimising interventions such as exercise and self-management rather than continuing to study something like acupuncture that lacks a plausible mechanism of action and shows little evidence of benefit.

Last updated 23/02/23