Most trials of alternative treatments are fundamentally flawed: an example based on paediatric osteopathy

Much of the research published on alternative therapies is of very poor quality and is often focused on trying to market a therapy rather than actually assessing whether it has benefits or not. There are many issues with the trials conducted in this area and in this post I would like to focus on two key problems:

1. Trialling a treatment for which there is no plausible way it could provide any benefit for the given condition.
2. Trying to show efficacy from a small pilot study.

Getting these two points right is, in my view, beginner level study design. They are often done incorrectly in trials of alternative therapies. Paediatric osteopathy, much like paediatric chiropractic, is mired in pseudoscience and wildly unrealistic treatment claims. I’d like to use this trial of paediatric osteopathy as an example of the points above.

Overview of the study

The title of the study was “Efficacy and Feasibility of an Osteopathic Intervention for Neurocognitive and Behavioral Symptoms Usually Associated With Fetal Alcohol Spectrum Disorder”. The intervention in this case was 4 weeks of osteopathic manipulative treatment (OMT). It was a pilot study with just 32 patients split between intervention (OMT) and a control (standard support measures).

Plausibility

Looking at the first criteria I highlighted above, is there a plausible way that osteopathic manipulative treatment (OMT) could influence neurocognitive and behavioural symptoms? The short answer is a very resounding “no”. There is just no reasonable way that performing manipulations could have an impact on behaviour or the functioning of the brain in any health condition.

It is, however, worth looking a bit more closely at the treatments used in the osteopathic approach to understand just how ridiculous they are. According to the paper “Somatic dysfunctions were detected by physical examination, based on tissue texture changes, asymmetry, limitation in normal range of motion, and tissue tenderness parameters (TART), which guided the osteopathic evaluation and OMT intervention”. Firstly, some degree of asymmetry is normal in everyone and, in spite of claims by many chiropractors and osteopaths, there is no evidence that trying to “correct” this provides any benefit. Secondly, what possible relationship could a limitation in range of motion or tissue tenderness have to behaviour or brain function? This just makes no sense. They go on to say “Using OMT techniques, the identified somatic dysfunctions were corrected one by one in the whole body”. Again, how could any of this possibly have any effect on the symptoms they are trying to treat?

Efficacy and pilot studies

The purpose of a pilot study is to assess feasibility of a treatment approach and understand associated parameters in preparation for a fully powered efficacy trial. A pilot study should NEVER be used to determine efficacy of a treatment. In trials of alternative therapies this very basic rule is often broken with many pilot studies trying to claim that the tested treatment is “effective”. This is exactly what we see with this study: “The main primary objective of this pilot study was to evaluate the efficacy and feasibility of a 4-week planned OMT intervention” (emphasis mine). No, no, no and again no! The authors recognise that this is a pilot study but then claim the primary objective is to evaluate efficacy. If the people conducting a pilot study don’t understand the purposes of such a study they probably shouldn’t be doing the research in the first place!

Other issues

Besides these two very fundamental issues with this study, there are other problems too. For instance, they are trying to develop a treatment for fetal alcohol spectrum disorder. However, in this study they recruited patients that DO NOT have this condition. The patients recruited were “Children aged 3 to 6 years without a FASD diagnosis but with symptoms usually present in FASDs”. If you want to see whether a treatment works for a particular health problem then a pretty basic starting point is recruiting people with that condition.

These sorts of very basic problems are all too common in studies of alternative therapies.

Conclusion

Like many alternative therapies, paediatric osteopathy is mired in pseudoscience and misleading claims. The research conducted into these approaches is often focused on marketing the particular pet therapy rather than genuinely researching it. Using a trial of paediatric osteopathy as an example, I have highlighted two very basic flaws that are common in many studies of alternative therapies:

1. Trialling a treatment for which there is no plausible way it could provide any benefit for the given condition.
2. Trying to show efficacy from a small pilot study.

Studies like this are a waste of time and money and, in my view, are unethical because they involve testing biologically implausible treatments on unsuspecting members of the public. The results from these sorts of trials shouldn’t be trusted and they certainly shouldn’t be used to inform treatment choices.

Zero out of twenty chiropractors compliant with regulatory advertising guidance

Misleading claims in the UK chiropractic profession has been a serious problem for many years and, in spite of plenty of guidance from the Advertising Standards Authority, continues to be a major issue. There are many different types of misleading claims, ranging from the treatment of babies to giving the impression they are doctors and even claiming to be able to boost the immune system. On 3^rd November 2021 the UK chiropractic regulator, the General Chiropractic Council (GCC), published a new advertising toolkit for chiropractors. It’s a pretty good guide but there is nothing new there that chiros shouldn’t already know. They should already be compliant with everything that’s in there. To test the current compliance level, I assessed a random sample of 20 London-based chiropractors against the new toolkit. Zero out of the 20 were compliant! There were a staggering 18 who I believe would warrant regulatory complaints and 2 who were “not that bad”. Although I’ve been tackling misleading claims in chiropractic for many years, I was surprised just how bad these results were. I really thought there would have been a few who were fully compliant and a reasonable number of the rest wouldn’t have been that bad. I was wrong. The amount of pseudoscience and misleading claims was shocking. Let’s look in a bit more detail.

Selecting the random sample

In order to select the random sample of 20 chiropractors, I used the “Find a chiropractor” tool on the General Chiropractic Council’s website. I then entered “London” in the “City” box and clicked on “See Search Results”. I then selected the first 20 chiropractors returned by the search, excluding the following:

• Unable to find a website or website only has contact info with no description of practice.
• Chiropractor listed as “not practicing” on GCC website.
• Chiropractor is purely an academic and does not treat patients.

Now you too can easily repeat my search and see for yourself just how bad these chiropractic practices are.

Assessing the chiropractors

I used the GCC’s advertising toolkit to produce a list of criteria against which to assess the chiropractors, including:

• Advertising for conditions outside of those that chiropractors are allowed to advertise to treat.
• Anti-vaccination stance
• Subluxation or “correcting alignment”
• Claiming to “treat the root cause”
• Suggesting that long-term care can prevent illness
• Claiming that chiropractic treatment can improve immunity
• Giving the impression that they are a doctor or equivalent to a doctor

Results

Out of 20 chiropractors assessed there were zero who were fully compliant. The most common issue was advertising to treat conditions outside of the “allowed” list. Every single chiropractor was in breach of this, some to a greater extent than others. Other breaches that were common included:

• Subluxation or “correcting alignment” – 55% (11 out of 20) made this claim
• Claiming to “treat the root cause” – 50% (10 out 20) made this claim
• Suggesting that long-term care can prevent illness – 40% (8 out of 20) made this claim
• Giving the impression that they are a doctor or equivalent to a doctor 45% (9 out of 20) made this claim

Thankfully, there were no chiropractors who made openly anti-vaccination claims on their website. However, it is likely that this would be sufficient to result in regulatory action and it’s therefore perhaps not unsurprising that no chiropractors would openly admit to being anti-vax. As to what happens when they see their patients, that is much harder to assess.

Some of the claims made by chiropractors were pretty extreme. For instance, the lead chiropractor at one practice said that he had a “special interest and training in neurology” and claimed to be able to treat “neurological conditions”. Now, chiropractors are not doctors and it should be very obvious that treatment of neurological conditions is way outside of their expertise.

Another example is these claims relating to babies from Northcote Chiropractic:

Fig 1. Very misleading info about “check-ups” on babies from Northcote Chiropractic

It’s all complete nonsense, but what parent reading this wouldn’t be scared into taking their child in for treatment?

Target Health Chiropractic repeated entirely false claims from the infamous Joseph Mercola:

Fig 2. False nutrition claims from Target Health Chiropractic

I have previously made an Advertising Standards Authority (ASA) complaint about one of the chiropractors in this sample, Putney Chiropractic. This doesn’t seem to have resulted in much improvement as they still make a whole range of misleading claims.

Overall, these results are awful. Not only were all chiropractors in breach of guidance from the regulator but the level of pseudoscience and misleading claims were extreme in a number of cases.

Conclusions

Some chiropractors claim that the problems within their profession are confined to a minority. What this survey shows is that this is very clearly not the case. Out of a random sample of 20 chiropractors, there were zero who were fully compliant. A significant proportion of this sample of chiropractors gave the impression that they are equivalent to a doctor (they aren’t) and more than half treat “subluxations” or “correct spinal alignment”. This isn’t supported by science or evidence.

I was genuinely shocked by just how bad the misleading claims were from this sample of chiropractors. Whilst a sample of 20 doesn’t mean that every UK chiropractor makes misleading claims, it does give an indication of just how much of an issue this is within the profession. Based on the results of this sample, I can only give the following advice: Please don’t visit a chiropractor.

Chiropractor in breach of chiropractic code of practice but found “not guilty”

The Professional Conduct Committee (PCC) of the General Chiropractic Council (GCC) have recently published their decision in the case of chiropractor Arleen Scholten. This was the very sad case of John Lawler who died after chiropractic treatment. The committee concluded that the allegation of “unacceptable professional conduct” was not well founded. Their primary rationale for this decision was that, although there had been breaches of “The Code: Standards of conduct, performance and ethics for chiropractors”, that these breaches were due to Mrs Scholten’s state of mind at the time rather than deliberate intent to be inaccurate or misleading.

In reaching their decision, the committee seem to have given great weight to:

1. An acute stress reaction being the source of the numerous inaccuracies in Mrs Scholten’s reporting of the events and records.
2. Testimonials from patients and colleagues.

They seem to have not considered, or inadequately considered, that:

1. She lacked even basic first aid knowledge and as such provided totally inappropriate care during the medical emergency that occurred.
2. She changed her position multiple times with regards to what she claimed to have said to the ambulance crew.
3. She had previously (probably for many years) been deceiving the public by referring to herself as “Dr”.

Let’s look at each of those items in a little more detail.

Acute Stress Reaction

The committee received advice from two expert psychiatric witnesses and on the basis of this advice concluded that Mrs Scholten had been suffering from an “Acute Stress Reaction”. The committee considered that this provided an explanation for Mrs Scholten’s inaccurate reporting of the treatment provided to the 999 call handler, the ambulance crew and inaccurate records in her own clinic notes about the treatment provided. However, there are a number of things that don’t seem to stack up here particularly with regards to the sequence of events:

1. When the ambulance crew arrived, they both described Mrs Scholten as being calm with one of the crew stating that she “handed over the patient and what she had found in a calm manner” and the other saying that she “was reasonably calm”.
2. Immediately after the ambulance had left, the clinic receptionist said that “Mrs Scholten was crying and shaking, she was in a state of shock.”
3. Mrs Scholten then went on to treat other patients. According to her receptionist “I asked Mrs Scholten if she wanted to see a couple of patients who had arrived. After about five or six minutes she went to see them”.

Now, does somebody go from a state of being unable to accurately explain the treatment she has provided to an ambulance crew, to crying and shaking, to then treating further patients in what appears to be (according to the accounts) a relatively short amount of time? If Mrs Scholten was so stressed, how could she contemplate treating further patients? Wouldn’t it be a duty of care to cancel those patients, recognising that she was unable to provide suitable treatment?

Testimonials from patients and colleagues

The committee seem to have given great weight to testimonials from patients and colleagues and they state “Indeed, the Committee had never before encountered such an impressive collection of character evidence, which it considered particularly noteworthy”. Now, even the very worst healthcare professionals can usually find people who are willing to provide positive testimonials. For instance, I know someone who was once a patient of the notorious Harold Shipman who said that they had often had a good experience seeing him as a GP! It is a serious mistake for any conduct committee give much weight to a set of self-selected testimonials from someone they are investigating.

Interestingly, several of the testimonials are from patients who went to Mrs Scholten for regular treatments over many years. This is in itself a concern, and would suggest inappropriate care, as there is no benefit to regular or routine chiropractic treatment.

First Aid Training

At the time of this tragic incident there was no specific guidance provided to chiropractors on the need for them to have first aid training. You might reasonably expect that a healthcare professional who is allowed to treat patients fully independently would be trained to handle any emergencies resulting from such treatments. There was, however, no such requirement. In fact, the General Chiropractic Council brought in new guidance in March 2020 relating to First Aid presumably in response to the very basic errors made by Mrs Scholten in this case. Interestingly, the GCC largely absolves themselves of any responsibility for deciding what First Aid training is appropriate and instead push that decision onto the individual chiropractors. This seems likely to result in very different approaches and a lack of consistency across the profession. This is a potential risk to the public, the very thing the GCC are there to protect against. As Edzard Ernst reports in one of his posts on this case, Mr Lawler would probably have survived if Mrs Scholten had not provided the entirely wrong first aid care.

Changing her statement

In a statement to police several days after the incident, Mrs Scholten claimed that she told the ambulance crew: “I explained that following the use of the drop table [Patient A] had reported loss of sensation in his arms and I confirmed the sequence of events.” However, she later changed her position on this and the GCC report states “It was subsequently no longer Mrs Scholten’s case that she informed the paramedics of the use of the drop technique”. In later written observations to the GCC, she then claimed that the did not classify the Thompson Drop technique as a manual adjustment and she did not “therefore accept that I misled the paramedics by stating that I did not carry out any manual adjustments”. With all of these changes of statements I think it is reasonable to question the reliability of Mrs Scholten’s testimony.

Use of “Dr”

I have previously written about how, at the time that Mr Lawler went for treatment, Mrs Scholten marketed herself as “Dr”. This is a widespread issue within the chiropractic profession. However, a chiropractor who misleads the public about their professional status doesn’t suggest someone who is honest and trustworthy.

Conclusions

Of all of the professions that offer “alternative” treatments in the UK, there are two that have statutory regulation. Those are chiropractors (regulated by the General Chiropractic Council) and osteopaths (regulated by the General Osteopathic Council). The purpose of this statutory regulation is supposed to be to provide independence from the professions and therefore more effectively protect the public. Here we have a chiropractor who deceived the public by referring to herself as “Dr”, repeatedly changed her statement on key facts within the case and provided totally inappropriate first aid care after her patient suffered a severe reaction to her chiropractic treatment. The regulator has decided that this is all okay, does not amount to unacceptable professional conduct and that she can continue to practice without any sanctions. Whilst decisions in cases like this are clearly very complex, it is hard to see how the GCC is fulfilling its primary regulatory duty of protecting the public by reaching this decision. I would suggest that this decision undermines any remaining trust that the public could have had in the chiropractic profession.

Are osteopathy and chiropractic treatments really as safe as they are claimed to be?

I have seen many chiropractors and osteopaths claim that their treatments are “very safe” but is that really accurate? This is a complex question without a straightforward answer. A survey of UK osteopaths helps to shed some light on this important question. Whilst that survey focuses specifically on osteopaths, the situation with chiropractors is likely to be similar as many of the same treatments are used by both groups of practitioners.

Treatment effectiveness and risk / benefit analysis

When assessing the suitability of treatments, it’s important not to assess safety in isolation but to consider it in the context of the potential benefits of a treatment. A treatment that has high risks might still be the right option if the benefits it provides are also very significant. Similarly, a treatment that has limited benefit might not be suitable even if it carries low risks. What matters is the risk / benefit analysis. The primary treatments used by chiropractors and osteopaths (manual therapy) have zero or very small beneficial effects. So, when assessing risk / benefit even a low level of risk should probably result in a recommendation against these treatments.

Serious adverse events

In the above survey, there are some important findings related to serious adverse events:

• 4% of patients reported experiencing “temporary incapacity or disability that they attributed to their osteopathic treatment”. 10 of these patients were interviewed, 2 of whom described “experiences that were characteristic of a serious adverse event”.
• “Serious adverse events including severe new symptoms, the worsening of existing symptoms leading to hospital referral and/or permanent disability or incapacity or death were reported by 12% of osteopaths” over their career. 4% of osteopaths reported a serious adverse event in the past year.

Taken together the above statistics suggest that serious adverse events from osteopathy (and probably chiropractic, physios using manual therapy, etc) treatments are not frequent but they do happen to a reasonable number of patients. This is not good and it would not be appropriate to describe such treatments as “very safe”.

Consent

Consent is an important part of any treatment by a healthcare professional and falls under the Health and Social Care Act 2008. In the above survey, patient reports of receiving consent was lower than that claimed by osteopaths. Only 36% reported receiving information about risks and 38% about alternative or no treatment options. This is awful and is a serious ethical issue.

The report provides some insight into why informed consent may be so lacking, such as “Some osteopaths expressed concern that giving information about serious risks may cause stress and prevent patients from gaining the best outcomes”. What seems to be underlying these concerns from osteopaths is a fear that patients will decline treatments if informed about the risks. That is the whole point of the informed consent process!

Conclusions

The safety of treatments provided by chiropractors and osteopaths is an often debated topic. Many practitioners claim that their treatments are “very safe” without providing any data or evidence to support these claims. It isn’t appropriate to use a single survey of UK osteopaths to try and reach definitive conclusions about the safety of these treatments. However, this survey does provide some useful insights. We already know that the treatments from chiropractors and osteopaths are of limited therapeutic benefit. Based on the information from this survey, as well as numerous published case reports, these treatments do appear to carry risks of severe adverse events. Importantly, patients often appear not to be informed of these risks or the potential of alternative options before treatment is undertaken.

If a drug had such limited benefits and carried risks of severe adverse events it is unlikely that it would ever be licensed. Applying the same approach to chiropractic and osteopathy treatments, I would suggest that you think twice before visiting one of these practitioners.

Yet another negative study of chiropractic care for infantile colic: It is time to stop these inappropriate treatments on babies

Of all of the different patient groups that chiropractors and osteopaths treat, the one that concerns me most is their treatment of babies. Chiropractors and osteopaths continue to offer their treatments for a range of conditions in babies in spite of no good evidence that they help with anything. They also continue to conduct research to try and show that these treatments “work” even though in many cases there is no plausible mechanism by which they could possibly do so. A recently published trial looked (again) at chiropractic care for infantile colic. This was yet another negative study. I would argue that it’s high time that research into chiropractic and osteopathy treatments for babies was stopped. In my view it’s unethical to continue to expose this vulnerable patient group to these treatments given a) the lack of a plausible mechanism of action for these treatments and b) the very limited training that chiropractors and osteopaths have in the treatment of babies, which are a unique patient group.

The study

The study sought to assess the effectiveness of chiropractic care for infantile colic with parents blinded to treatment allocation. Both groups attended clinic twice a week for two weeks. The intervention group received chiropractic care and the control group was not treated. To maintain parent blinding, parents were not present in the treatment room. 185 babies completed the trial (96 in the treatment group, 89 in the control). Duration of crying was reduced by 1.5 hours in the treatment group and 1 hour in the control. The difference was not statistically significant. Let’s be clear, such a small difference in crying time means that the treatment didn’t show any effect and this is a negative trial. (The reduction in both groups is probably due to the tendency for colic to just improve of its own accord over time). However, the authors go on to suggest that further research should be undertaken “to investigate if subgroups of children, e.g. those with musculoskeletal problems, benefit more than others from chiropractic care”. I disagree with this on a number of points:

1. There is no indication that colic has a musculoskeletal origin so there isn’t a good rationale for the suggestion that these treatments might benefit crying time in babies with musculoskeletal problems.
2. These treatments carry risks.
3. Chiropractors have very limited training in the treatment of babies and lack the specialist knowledge required for this unique patient group

Risks associated with these treatments

An extensive review in Australia looked at chiropractic treatment of children. It stated that “it is difficult to draw conclusions about the safety and effectiveness of spinal manipulation in children” and went on to say “Nonetheless, it is clear that spinal manipulation in children is not wholly without risk.”

Conclusions

This is yet another negative study of chiropractic care for infant colic. The evidence is really pretty clear: these treatments aren’t effective. The situation is the same for chiropractic and osteopathy treatments of other infant problems: there is no good evidence that these treatments benefit any condition. Given the potential risks associated with some of these treatments and the lack of expertise of the practitioners with the treatment of babies, there is no good reason to carry out any further research on these treatments in this vulnerable patient group.