It’s time we redesigned the clinical trial
pharmafile | May 16, 2011 | Feature | Research and Development | clinical trials, randomised controlled trials
Who can remember a time when the pathway from laboratory bench to marketed product didn’t include clinical trial phases I to III? I can’t, despite my increasingly grey hair. The conventional stepwise process has been embedded for so long that it has almost assumed the aura of a religion.
Heretics challenge it at their peril. Regulators don’t overtly force us into what might appear to be a procedural straitjacket, but it seems that we would need compelling reasons to depart from it.
Out of time
Yet it is clear that moving from phase I to phase II and onwards isn’t a strictly linear path. Phase I clinical studies, and even non-clinical ones, may well be conducted in late development.
I have found myself thrown into these (and also further animal studies) at the end of phase II, when clinical findings exposed gaps in the earlier data. The lesson here is that research of any kind is uncertain, and the researcher needs to be flexible. This sometimes causes confusion between regulators and sponsors. I well remember my carefully constructed time plan being thrown awry when I could not get the expedited approval of a phase I study that I was expecting.
This was because the regulator said it wasn’t phase I, because we were doing it alongside phase III studies. Well the participants were healthy volunteers and we were conducting it in a phase I unit, but there was no argument to be had.
The uncertainty which is endemic in science is well reflected in drug development. Not only do we have the inescapable uncertainty attending the discovery of new knowledge, but in clinical development we have a high degree of operational risk. A project manager new to the pharmaceutical industry was once heard to exclaim: “Do you mean to say that you actually can’t predict with any certainty when you will enrol all the patients you need?” He was used to ordering a number of bricks and they would all turn up on a fleet of lorries on the appointed day.
If they didn’t, he had words to say to the contractor.
Wake up and smell the….(it’s not coffee)
Given these risks and uncertainties, it is hardly surprising that efforts are being made to break the established mould, or at least to melt it a bit and get it into better shape. I think these efforts fall not very tidily into two main areas; changing the scientific methods, and changing the management approach. There can be no doubt that change is needed, and we have known this for a very long time. Despite great advances in technology, clinical development takes the same time as it did 10 years ago (see CMR International’s R&D Factbook). Only one in five drugs launched recovers its development costs. Innovation has taken a dive. Drugs continue to fail in phase III, often when they don’t meet efficacy targets. Major drugs have been withdrawn from the market because of safety concerns.
So let’s look first at the science. The established randomised controlled trial (RCT) is the foundation stone of the current drug development process. It is very good at telling us whether there is something going on, but it has limitations. These were eloquently described over two years ago by NICE chairman Professor Sir Michael Rawlins in his Harveian Oration at the Royal College of Physicians in 2008.
The Oration is recommended reading (visit the RCP Bookshop website). The thrust of Rawlins’ argument, was, however, widely misinterpreted – he wasn’t dismissing clinical trials (as the lay media thought he was), he simply set out where they were inappropriate, and where they were limited. A key limitation is their essentially binary nature, in that we tend to regard the outcome as ‘passed’ or ‘failed’.
The real world is not like that. RCTs are not great at quantifying the effect size, and hence the treatment’s impact on normal clinical practice. For this reason, there has been much emphasis in recent years on ‘pragmatic’ trials which have less stringent selection criteria, and attempt to model the real world more realistically.
Are short cuts possible – and safe?
But what if, in this brave new world of scientific innovation, we decided to skip the restrictive and unrealistic RCT and rely on pragmatic trials and observational data? At present that would not get us far, as the regulators are still wedded (or I could say welded) to the RCT. But in any case such a pathway is fraught with danger.
‘Real world’ studies (some are trials, some are surveys and other uncontrolled ‘experiments’), are much less refined tools. They are more likely to give us a misleading result, either a false positive or a false negative. It seems that, notwithstanding some innovations that I will address shortly, the RCT will remain the basis of our knowledge regarding a drug’s efficacy.
If they give us a positive result, the next stage should be to test generalisability with less rigorous but more realistic studies. At present these generally are allocated to phase IV, but I can foresee the regulators looking at them with more interest.
Back to the 18th century
The RCT is the embodiment of the classical ‘frequentist inference’ approach to experimentation and statistics. There is, however, an alternative approach – Bayesian inference – but this has not taken hold in the same way in clinical trials. Thomas Bayes was an 18th century Presbyterian minister who, like many clergymen of the day, had enough spare time to pursue more interesting things, in his case mathematics. Quite simply, classical methods do not give us direct insight into a key question – the probability that one treatment is more effective than another. Bayesian inference can do this, but requires quite a different approach and mindset.
The basis of the idea is that of a comparison – not between treatment and comparator – but between prior and posterior probabilities.
‘Prior’ means before the intervention, and more correctly is a distribution of possible values. The experimenter would get this from previous experiments. Have you spotted the pitfall yet? For new drugs, there is of course no previous data. So Bayesian methods couldn’t entirely replace the classical methods, but in some settings they can be extremely useful.
The theorem is quite simple then, but its implementation is rather complex, which might account for its limited uptake in medical science. Use of Bayesian inference may well have been held back by a lack of adequate computing power. Analysis may take thousands of iterations, and I might worry about validating the software for it, but no doubt the experts have that covered. One advantage is that, as knowledge advances, of course the prior distributions become more concrete, so the method works well in an adaptive process. Bayesian designs are also better at quantifying effects.
Adaptability may be the key
But what do you do before you have any prior knowledge? This is an obvious gap, so a compromise might be to carry out RCTs at the start and then convert to Bayesian designs, using the data from RCTs to prime the pump. I just mentioned adaptive processes, but these shouldn’t be confused with the Bayesian approach.
The key concept of adaptive trial design is in accumulating data to adapt and refine the trial while it is ongoing, but without undermining its validity or integrity. In reality we have had adaptive trials for very many years. In oncology studies, treatments usually have to be modified according to how the patient responds, i.e. they adapt treatment according to the response.
Another example is toxicity studies, in which subjects are enrolled in groups on an ascending scale of dose, and the decision to proceed to the next level depends on how many subjects exhibit toxicity. These are simple examples, and adaptive trials have proliferated into a range of sophisticated designs – for which Bayesian statistics are particularly helpful.
The RCT is a very simple thing to understand, and fairly easy to model in a management environment. I often encounter heated arguments in forum discussions about project management, as to whether planning should be detailed or top level. I have always tended towards the detailed side, on the basis that detail is easy to delete if you find you don’t need it, but very hard to insert if you missed it at the beginning. As drug development is becoming more adaptive, more ‘real world’, and less predictable, it is going to be more difficult to model every task in a clinical trial, never mind in the whole programme.
Ironically, I wonder whether this might be an advantage. Micro-management is the bane of project managers, and generates vast amounts of useless work.
The problem emanates from a focus on tasks and not on outcomes. Whatever the scientific approach, the desired outcomes are going to be the same. The target product profile would have been defined at the outset (but of course may well be adapted), so the criteria for assessing efficacy and safety will be clear. With a wider range of approaches available, each generating a different set of tasks, the only common thread will be outcomes (embodied in deliverables).
There is some evidence that CROs are moving in the direction of outcomes-based contracts.
Meanwhile, such potential diversity throws much more emphasis on the need for formal project definition, and effective choice of methodology.
How safe is ‘safe’?
I should conclude with some thoughts on safety assessment. RCTs are really very poor for this, as they are powered to test efficacy and not safety. They are just not big enough to give us a good grasp of the less frequent safety signals.
Yes, it seems possible that obvious signals in phase III were missed or ignored in certain recent cases, but in general only post-marketing surveillance can build up the picture we need.
The less conventional approaches I have touched upon here are not likely to have much impact on this, if it is really a matter of sample size. It is an issue that needs to be on the agenda for every drug regulator.
Related Content
Vesper Bio reports positive topline results for dementia candidate
Vesper Bio, a clinical-stage biotech developing novel oral therapies for neurodegenerative and neuropsychiatric disorders, has …
Von Willebrand disease – increasing awareness and access to vital care
Pharmafile talks to Anthea Cherednichenko, Vice President Franchise Head Haematology and Transplant at Takeda about …
Rethinking oncology trial endpoints with generalised pairwise comparisons
For decades, oncology trials have been anchored to a familiar set of endpoints. Overall survival …
