Is clinical research management evidence-based?
pharmafile | September 29, 2014 | Feature | Manufacturing and Production, Medical Communications, Research and Development, Sales and Marketing | Les Rose, Six Sigma, clinical trials, evidence, neuro-linguistic
Those who follow my columns will know that I am in the habit of starting with reminiscences. It’s something that comes with the grey hairs. So I make no apology for once again taking you back to a time most of you won’t remember, and before a lot of you were born. I am talking about the late 1970s and 1980s.
The tradition when I first entered clinical research was that it had no credible career pathway unless you had a medical degree. All of the management positions were occupied by clinicians, or occasionally PhDs. I can’t think of any senior managers who were not clinicians at that time.
This hegemony took a very long time to erode, and even much later the prejudice continued even if the practice was weaker. About 10 years ago I was sitting at lunch during a pharmaceutical conference, engaging in the usual pleasantries to my neighbour.
He had assumed that I was a medical doctor like himself, and when I corrected this observation his head swivelled round to his other neighbour and he never said another word to me.
The curious thing, was that while all the top jobs went to doctors, they were not generally highly trained in management, if at all. When I started running training courses in project management, the delegates spent much of their time complaining about the doctors who managed them.
Of course, those doctors mostly didn’t attend such courses, they just sent their people on them. Now let me clarify that I’m not prejudiced against medical doctors (my son is one), but whatever job someone is given, they need to be competent to do it.
Such competence is rarely in-born, hence the need for training. So over the years there has been an increasingly intense focus on how the hugely challenging process of clinical research can be more effectively managed, with new techniques coming in from other industries.
Out of control
Let us not underestimate the massive problems that clinical trials present. As well as the technical risks of testing new treatments, there is the question of control. The paradox is that as the scale of trials increases over time, from small Phase I to huge Phase III, we are more and more forced to farm out the work to people over whom we have less and less control.
Managing contract research organisations is not easy, but we are lost without effective investigators in clinical sites. These are often highly dispersed, with most investigators not dedicated to our trials but having a main job as a clinician.
When I explain this to project managers from other industries they are astonished. They are used to having clear and enforceable procurement plans that minimise the risk of the steel girders arriving late or the concrete contractor’s team going on strike.
Delegating critical work to people we can’t directly control is an alien concept to them. So I am not surprised that pharmaceutical companies have explored a wide range of management disciplines to try to stem the tide of escalating costs and time scales in R&D.
The big question is, are they getting value from these ideas? I don’t have space here to examine every research management tool, so I’ll confine the discussion to some that I have encountered over the years in the pharmaceutical industry. I’ll be wearing my critical thinking hat, and the starting point is my old favourite, classical project management.
From its roots in Henry Gantt’s bar charts 100 years ago, project management has proliferated and diversified. We now have agile, critical chain, lean, and extreme flavours, as well as of course the vaunted PRINCE2.
There is quite good evidence that it’s highly effective in its core industries of construction and engineering, and there is no logical reason why it should not be equally effective in clinical research. But it’s actually very difficult to find solid evidence of that in terms of comparative metrics.
Indeed, the most basic requirement, that of measuring what is going on, often does more to confound the manager’s efforts. I well remember asking a company how much benefit they got from a very expensive critical chain software package, and they didn’t know because they had never measured anything before they implemented it.
Metrics change over time anyway, so establishing cause and effect can be challenging. Evidence in favour of project management in healthcare research is rather more qualitative. There are studies that report favourable opinions from project team members, which must be a benefit in itself (motivation is a high priority for the project manager).
But after 25 years in training and consulting, I would have expected more quantitative evidence, and the message here is the old one that to have control you have to measure what you are doing.
Critical chain
Various techniques have been ‘bolted on’ to classical project management, and some are rationally based while others may not be. I have mentioned critical chain method, which should have a close fit with clinical research as it is closely focussed on resources and uncertainty.
Nobody has too much money or too many people, so modelling a project on resource constraints makes a lot of sense, and any research project must have a great deal of uncertainty embedded in it. It is claimed that completion times can be 15% shorter one year after implementing critical chain method, with key tasks often 35% shorter.
Much of the benefit stems from a major change in the way people are managed, in particular by front-loading the planning effort. Team meetings at the planning stage are much longer than usual, with better-defined objectives.
Leaning towards Six Sigma
The ‘lean’ concept is inextricably associated with Toyota, and is said to have been pivotal in building the company into the world’s largest automotive manufacturer. The core principle is the reduction of waste, and while its roots are in manufacturing, it is readily translated into the world of projects.
It has also become closely linked to Six Sigma, developed by Motorola and focussing on reduction of process defects and variability. Both techniques interface with total quality management. A substantial literature has built up concerning the application of these concepts to clinical research, and some dramatic improvements in cycle times have been claimed.
For example one study reported a 70% reduction in time for case report form data entry, using a technical process refined using Six Sigma. Another reported a major improvement in Phase I trial completion time using a varied set of new processes, plus redesigning some standard ones.
But Six Sigma is not without its critics, some of whom consider that it is too narrowly focussed on existing processes and can stifle creativity. For this reason it might not be well suited to research projects. It is also said to be over-reliant on statistical tools for reducing variation in existing processes, diverting effort from creating robust processes in the first place.
However, most relevant to my theme here is that Six Sigma may not be well evidenced. Most of the evidence for its impact comprises case studies published without academic rigour, as exemplified above.
On the grand scale, while it may achieve dramatic improvements in problematic processes, there seems to be little evidence that it transforms an organisation’s overall effectiveness. Nevertheless, I am intuitively attracted to techniques that reduce waste and error.
Many years ago my team implemented a quality management system compliant with ISO9001, and my personal experience is that the attitude of the work force to quality changed for the better. Again it often comes down to changing behaviours, the most difficult thing you can do but usually the most effective.
In contrast, some other techniques have become associated with project management in the absence of any rational basis. One of these is neuro-linguistic programming (NLP), which is encouraged by some professional bodies such as the Association for Project Management.
On the surface, NLP looks sensible. If it’s possible to analyse the skills of successful people, then it should be possible to replicate those skills in others. The problem is that the proposed mechanistic basis of NLP has not been verified.
In short, it is pseudoscience. Its inventors, Richard Bandler and John Grinder, brought together a very wide range of ideas from such fields as hypnotherapy, linguistics, mathematics, philosophy of science, and formal logic.
But they have been criticised as misrepresenting, misunderstanding and misinterpreting many of these ideas, few of which have any real relevance to their conclusions. None of this seems to have stopped NLP from developing into a major business sector, even to the extent of making therapeutic claims for various illnesses.
In this respect it is similar to Scientology. Controlled trials have confirmed neither the underlying assumptions of NLP nor its claimed effects. These examples illustrate some key messages. It is very easy to be seduced by the next enthusiast for some new management technique. It may all sound very logical and self-evident, but does it work? This is the acid test question that we all have to ask – and we have to demand robust evidence.
As we have seen, case studies may read well but they are all too easily selected from a pool of outcomes. You won’t hear about the failures. I believe that there is very little that can’t be tested by a controlled trial, and that’s what we should expect.
My next message is that it’s always better to do simple things well than to try complex things and fail. If you are convinced that critical chain method is for you, make sure you have all the resources you need to implement it fully. You will waste time and money by doing a partial job.
I have been quite critical of management techniques for clinical research projects, but that’s not to say we should dismiss them. The process of getting new medicines to patients is so inefficient that complacency isn’t an option. I’m just alerting you to the need for critical analysis of the options you are considering.
Les Rose is a freelance consultant in drug development
Related Content
NeuroSense partners with PhaseV on upcoming phase 3 ALS trial
NeuroSense has announced that it has partnered with PhaseV for its upcoming phase 3 trial …
Nxera Pharma partner Neurocrine Biosciences enters phase 1 NBI-1117567 study
Nxera Pharma’s neuroscience-focused biopharmaceutical partner Neurocrine Biosciences has announced it has successfully started its phase …
Mirum Pharmaceuticals publishes phase 3 MARCH maralixibat data in The Lancet
Biopharma company Mirum Pharmaceuticals has announced data from its phase 3 MARCH-PFIC study which evaluated …
