Designing clinical trials for the real world
pharmafile | September 15, 2011 | Feature | Research and Development | clinical trials
A few years ago I was dealing with a big CRO which had been commissioned to run a pharmacokinetics (PK) study and to take it through to final report. The study wasn’t too much of a problem, but the report was.
Now PK studies can be complex. We had a barrow load of data on all the usual parameters, which we had to send to a pharmacokineticist to analyse into outputs that a general biomedical reader could understand. When the PK report came back, the plan was that it would go to the CRO’s medical writer for interpretation.
It seems that people interpret the word ‘interpretation’ differently. What happened at first was that the outcomes in the PK report were dropped into the results section of the clinical study report. Fine so far. But of course we also needed the discussion and conclusions. The first draft of the report came back, and when I got to the discussion I was smitten with déjà vu.
It was mostly bits of the results pasted in. Nowhere had the writer said anything about what the data meant. There followed a series of phone calls during which I had to coach the writer in the skill of interpreting what the data were telling us.
Over the years I have been guided by this question ‘What does it mean?’. Actually it takes me back to science practicals at school, when I was told that at the end of an experiment you should not be able to say ‘So what?’. This lesson has been a long time in the learning in clinical science, but maybe we are getting there.
We are moving on from reliance solely on the randomised controlled trial (RCT) for our evidence of effectiveness. But we can’t leave the RCT behind, and I’ll come back to that shortly.
A moving standard
The standards for reporting clinical research are now a bit different from what they were ten years ago. In some ways they are more demanding, and in others they are much simpler.
The excellent Bandolier website has for several years been condensing data to what it calls ‘bullet points’ (hence the name of the site).
Somewhat counter-intuitively, every summary is headed by the ‘clinical bottom line’. It is actually the headline, and states in a sentence (or perhaps three) what the findings mean.
This approach has been taken up by some of the major journals. The British Medical Journal now asks authors to submit a four-point summary, including how the findings will change clinical practice. This can be quite difficult to do – and I speak from recent experience.
Distillation of meaning
But to return to Bandolier, it is in a certain way the utmost distillation of meaning. This is because it concentrates on meta-analyses and systematic reviews. These are considered by most people in the field of evidence-based medicine as the pinnacle of the so-called hierarchy of evidence. But because such papers try to pull together all the existing RCTs on a topic, they can be quite demanding to read. Bandolier gets the whole thing down to one page. Needless to say, there are compromises, and if you don’t want to rely on Bandolier’s assessment of the methodology, you will have to read the whole paper. I am not too worried about that, as the site is run by independent academics and I think commercial or any other sort of bias is unlikely.
Knowledge is power – if it’s correct
But what are we really aiming at in this endeavour? Let’s take our cue from the BMJ’s question about impact on clinical practice. This question actually goes back at least a decade, to a time when the UK Cochrane Centre called for RCT results to be put into the broader context of knowledge in the field concerned.
Clinical practice is necessarily guided by current knowledge, or should be, so what we are really asking is how the new findings will add to that knowledge. This to me is a really interesting concept. What is accepted as current knowledge may not be right – in which case it would not be knowledge at all! Indeed there could be no progress unless there were scope for improving knowledge, when you think about it. How many of you had your tonsils removed as children? I escaped that abuse, but my younger sister didn’t. The received wisdom was that tonsillectomy would stop the problem of recurrent tonsillitis. Well it had to, as there were no tonsils to get inflamed. But RCTs showed that there was no effect on sore throats – the symptom, it was thought, of tonsillitis.
I believe you can still get billed about £1,500 by a private health company for the operation, but we now know what the meaning of the RCTs was – don’t bother.
The marketing of risk
This is an example of the meaning being quite clear, in terms of impact on the individual patient. In some cases it can be less so.
There has been a great deal of controversy over the value of screening for various cancers.
Let’s consider breast cancer. There is no doubt that regular mammography reduces the risk of developing cancer, but the big picture is far more complex than that. There is a significant risk of harm from unnecessary surgery when the screening result is a false positive. Also, the cost-effectiveness of screening is hotly debated.
But what does all this mean for the individual woman? This average woman, perhaps on the Number 9 bus, will just want to know what it all means to her. I have heard it said that, whatever the statisticians may say, screening must be good because it gives the woman more knowledge than she had before. Well most of the time it does, but in a small minority of cases the ‘knowledge’ is wrong. What is difficult to assess is the balance of risks – the risk of a false positive versus that of a false negative. It doesn’t help the average woman when the scientific view changes over time.
It is hard to get across the principle that science is not written in stone. By its very nature, it advances as more knowledge is uncovered.
This unsettles most people, who like to feel they can rely on the facts. It is therefore easy to see why those who claim to treat disease unscientifically have so many customers. How much more comforting it is to be told: “Ah, I see your chakras are blocked in the spleen area”, or “You have too much yin and not enough yang”.
In such cases there is no need to derive meaning from the claim, it is just accepted.
A pharmaceutical paradox
The paradox under which we labour in drug development is that we are always looking for new knowledge and better treatments, yet our working environment is staggeringly conservative. We are not early adopters of new technologies, and the evidence we generate about our medicines remains rooted in the RCT.
To be fair, this isn’t all our fault. We are heavily regulated, and the regulators expect us to follow tried and trusted pathways. For this reason, there are limitations to how extensively we can interpret and apply the data.
It appears that the RCT results for certain antidepressant drugs may not have much meaning for most of the patients who take them. This has been widely interpreted as saying that the drugs do not work, but they do. They worked quite well in the populations studied in the RCTs.
It’s just that in the real world there is a much wider range of patients – hence the need for an extended range of clinical evidence.
Pharma in the driving seat?
This is something that needs to be taken into account when completing the BMJ’s bullet point summary. The study will not change clinical practice if the population studied has little relevance to what the practising clinician sees.
It also needs to be considered by peer reviewers. It is no longer good enough to check that the methodology is sound, the statistics robust, and the results intelligible. Indeed it never was good enough, but we are only just realising that.
In this field, it’s an area in which the industry could be pushing the regulators, instead of the other way round. The healthcare payers are growing more canny by the day, so maybe we should engage with those payers at the outset, find out what’s meaningful to them, and design our studies to suit. When we have study designs which meet that need, we can then engage with the regulators, with a powerful argument that this is what really matters to healthcare providers.
Have I stood trial design on its head? I don’t think so, because trials are projects, and the first step of project planning is to define the value-based outcome. I think we need to define the meaning as received by the customer, up front and clearly. Perhaps a lot of this is already going on, but I don’t see it very often.
Les Rose is a freelance clinical scientist and medical writer.
Related Content
Vesper Bio reports positive topline results for dementia candidate
Vesper Bio, a clinical-stage biotech developing novel oral therapies for neurodegenerative and neuropsychiatric disorders, has …
Von Willebrand disease – increasing awareness and access to vital care
Pharmafile talks to Anthea Cherednichenko, Vice President Franchise Head Haematology and Transplant at Takeda about …
Rethinking oncology trial endpoints with generalised pairwise comparisons
For decades, oncology trials have been anchored to a familiar set of endpoints. Overall survival …
