Clinical evidence revisited
pharmafile | May 18, 2014 | Feature | Business Services, Manufacturing and Production, Medical Communications, Research and Development, Sales and Marketing | Les Rose, Lord Saatchi, Medical Innovation Bill, clinical trials
One of the most important words in the English language is ‘evidence’. It has had a seismic effect on the practice of medicine, which has shifted from an art to more of a science within not much more than a century.
Before about the middle of the 19th century, medical textbooks could only suggest treatments on the basis of testimony from doctors who had tried some medicine or other.
Clinical trials were virtually unknown, and the idea of separating opinion from evidence – or subjective information from the objective – had not yet arisen. This seems a little surprising, as the scientific method as we know it today had already existed for perhaps 200 years. The experiments of Newton and Hooke were no different in principle from the Large Hadron Collider.
Keeping in control
I make these points to emphasise that the controlled trial is not the preserve of medicine, but was well established by the time clinicians realised it was a good idea. There are those on the fringes of proper healthcare who like to claim that the controlled trial won’t work for their special cases. Indeed, there is an inexhaustible supply of wild claims from maverick alternative practitioners, who seem to discover on a daily basis some new fact about how our bodies work.
A cardinal feature is that, when a randomised controlled trial (RCT) seems to support their latest claim, they love it – but if it doesn’t then the reason is that their special treatment is so special that it’s outside the norms of science. You can see where this thought process is going. The scientific method is what has told us for 300 years how the universe works, and if it doesn’t work for a claimed phenomenon then perhaps there is nothing happening.
Where RCTs don’t work
But let’s take a step back, because I am not going to claim that RCT evidence is always required. A few years ago, the Christmas issue of the British Medical Journal carried a paper which emphasised the lack of RCTs testing the effectiveness of parachutes. This was seized upon by some alternative medicine supporters, who took it far too seriously and used it to denigrate the role of RCTs.
They had not spotted that the BMJ usually carries joke articles at that time of
year. There is, however, a serious point – and one which the authors were subliminally making – which is that RCTs are only required when the outcome being considered is in doubt. Nobody has done any RCTs on splinting broken limbs, because that practice is so obviously going to help a bone to heal straight and not bent.
A sense of balance
All of this, needless to say, has important ethical aspects, embodied in the concept of ‘equipoise’. When planning a trial, we should be able to say to the ethics committee: “Look, we really don’t know which of two treatments is going to be better, so we need to expose these patients to both in order to find out”.
If one of the treatment arms is placebo, then we should have no clear evidence that the study treatment is going to have any effect. In reality, if say we are planning a Phase III trial, we will have evidence from Phase II to suggest that it’s worth testing the drug in a bigger trial.
But we all know that drugs that looked promising in Phase II can fail to show anything after that, otherwise we would stop at Phase II and launch the product (if the regulators would let us). So we still have equipoise at the beginning of Phase III, in terms of the level of confidence that the drug will be effective in a clinical setting.
A world full of doubts
So back in the world of broken bones and parachutes, we are nowhere near equipoise with either, meaning RCTs would be unethical. The corollary is that the more doubtful the outcome, the more we will need RCTs.
Nowhere is this better illustrated than in public health. A lot of large-scale interventions are unfortunately poorly supported by RCT evidence, usually because RCTs can be very difficult to conduct in that setting. A good example of this is hormone replacement therapy for post-menopausal women.
For many years, it was widely accepted that HRT reduced the prevalence of cardiovascular disease, as well as ameliorating the symptoms of the menopause and preventing osteoporosis. But this assumption was based only on cohort studies. Eventually, a large RCT was conducted, the Women’s Health Initiative Study, which showed that HRT increased the risk of cardiovascular disease.
Not by enough to outweigh the benefits for osteoporosis, but now clinicians need to consider the cardiovascular risk of HRT in relation to other risks that a woman might have. The RCT turned out to be a more powerful tool than a cohort study, for detecting what was really going on with HRT.
Evidence-free health care?
What this means to me is that public policy on major health interventions needs to be based on the best evidence, and that where there is doubt, only RCT evidence will do.
The difficulty is that in many of these cases, the risks and benefits can be hard to detect, often because they are small and need very large sample sizes. In my last article, I mentioned the Breast Cancer Screening Age Extension RCT, and since then I learned that, even with a planned sample size of three million (the biggest trial in the world ever), it lacks statistical power to detect a difference in all-cause mortality between screened and unscreened women.
In that case, it would appear that there is nothing to detect, and the sponsors do not expect there to be any overall benefit of screening, so we have to wonder why they are doing the trial. The answer to that question may be more political than scientific.
False balance
In my first paragraph, I referred to the need to separate opinion from evidence. This is something that is very poorly understood by politicians, and by the lay media. Politics is heavily driven by public opinion, which is why political parties employ spin doctors and cultivate the media. Broadcasters have an obsession with what they see as ‘balance’, which means that for every interview on radio or TV there have to be views for and against the topic.
This ignores the possibility that one side may be 100% wrong, having based their arguments on either the wrong evidence, or (more usually) no evidence. But this commonly doesn’t matter to politicians who decide public health policy. Public opinion may be wrong because it has been misled by the media and, in turn, government policy might be designed to address public opinion, and be wrong as well.
“I’m dying – let me try anything”
Here is an example: I am continually hearing about desperately ill cancer patients who travel the globe in pursuit of a treatment that will save them. This usually involves enormous expense, not just for the travel but also (much more) for paying the various charlatans who offer what they claim to be innovative treatments.
In the UK, we now have an initiative to allow doctors to prescribe unorthodox treatments without fear of litigation for failing to follow evidence-based clinical practice. It is called the Medical Innovation Bill, and was conceived by Lord Maurice Saatchi who tragically lost his wife two years ago to ovarian cancer.
He now wants doctors to be free to try ‘innovative’ treatments if patients request them, and the bill is backed by the Department of Health, who drafted the text. You can read all about this bill on the DH website, and a lot of it surprises me.
Not least of all that the DH has said the need to document outcomes from these innovative’ treatments was deliberately kept out of the text, as it was thought to be a deterrent to trying something new. I am baffled that nobody at the DH spotted that this would fail to generate any evidence base for these treatments. It worries me that government could have such a low regard for evidence in healthcare.
Real innovation
I do not deny that early access to new medicines is an issue, particularly for seriously ill patients. The UK does not have a very good record in this area. But we already have mechanisms to address this, such as compassionate use protocols, and now the MHRA has launched the Early Access to Medicines Scheme.
What is important about these initiatives is that such medicines are not being offered to very sick patients without any evidence, but earlier in their development than is usual. In contrast, the Medical Innovation Bill doesn’t say anything about a minimal level of evidence required for a new treatment to be used, only that such use should be agreed by a multidisciplinary team.
The word ‘evidence’ does not appear in the bill at all, but ‘opinion’ does six times. Surely this places it firmly in the 19th century rather than the 21st?
I hope I have set out a moderate view of clinical evidence. RCTs are the acid test where we are doubtful about interventions, but they are not the only form of evidence. We may not always need them. I have previously addressed the need for ‘real world’ studies which can help us to generalise RCT evidence to normal clinical practice, and I haven’t changed my view on those.
But I am a realist; I accept that we all have opinions. We just need to remember that they are not the same thing as evidence.
Les Rose
Related Content
Vesper Bio reports positive topline results for dementia candidate
Vesper Bio, a clinical-stage biotech developing novel oral therapies for neurodegenerative and neuropsychiatric disorders, has …
Von Willebrand disease – increasing awareness and access to vital care
Pharmafile talks to Anthea Cherednichenko, Vice President Franchise Head Haematology and Transplant at Takeda about …
Rethinking oncology trial endpoints with generalised pairwise comparisons
For decades, oncology trials have been anchored to a familiar set of endpoints. Overall survival …
