Service Insight: Discovery metrics – creating the right balance
pharmafile | February 28, 2011 | Feature | Research and Development |Â Â Atkinson Cowan, Kinapse, Service Insight, metrics, research and developmentÂ
SERVICE INSIGHT
Metrics can be powerful tools for driving performance in organisations and have proliferated in the pharmaceutical industry over the last decade. However, their uptake has sometimes been resisted, particularly in Discovery organisations, where measurement of effort and output has been felt to be in fundamental conflict with the need for the creative, unconstrained culture required to drive innovation.
Experience seems to confirm these initial fears. Metrics have become over-abundant and often create an administrative burden that far outweighs any resulting benefits. Furthermore, they can incentivise behaviours and actions that are, in fact, at odds with the goals of the Discovery organisation.
Perhaps the best example of this is the delivery of poor quality compounds to development, which has resulted when the number of outputs produced each year has been the primary metric for Discovery.
Experience suggests that there are two major reasons behind the failure of Discovery metrics to yield benefits. They are:
• Difficulties in defining metrics which adequately capture the essential strategy and tasks of the Discovery organisation
• Failure to understand co-dependencies between metrics.
Defining ‘fit for purpose’ metrics
There are considerable difficulties in identifying metrics that enable managers to positively impact performance in Discovery. While the objectives of a Discovery organisation are easily articulated in simple terms, they can be challenging to translate into goals which are relevant to day-to-day activities. Objectives around, for example, ‘being innovative’ are often poorly defined or misunderstood, and there can be clear differences between the understanding of senior management versus that of a bench scientist.
Compound ‘quality’ presents similar problems, where in addition to internal differences of opinion, Discovery may measure quality in different ways to its customers in pre-clinical or early development.
In addition, commonly used metrics do not always enable proactive performance management at the operational level. In interviews, managers in discovery organisations identified a focus on lagging rather than leading metrics, and a lack of alignment in metrics between different operational aspects (notably alignment of human resource management with departmental objectives) as areas for improvement.
Finally, the long delay between activities that take place in Discovery and the market launch of the resulting drug means that Discovery managers need to rely on metrics which are, in essence, surrogate measures of future success – and these can be very difficult to define.
The challenge of co-dependencies
There is a fundamental tension in Discovery between the quality of the compounds that it produces and the time and costs incurred in their creation. If metrics focus on a single one of these dimensions, tensions emerge in the others – measuring time incentivises getting compounds into development faster and this compromises either – or both – of quality and cost. The temptation is then to measure these as well, and because additional tensions exist between and within each of these broad areas, there is a risk of developing an enormous cascade of metrics which, at best, fail to achieve their purpose and, at worst, incentivise counter-productive behaviours.
Getting the balance right
Despite these difficulties, it is clear that the proper identification and application of metrics can simplify the task of managing Discovery.
The first step in getting this right is to map how the strategic goals of the Discovery organisation translate into actionable objectives throughout the various functions. The figure above illustrates such a cascade and shows how the task becomes more difficult at the more detailed level of day-to-day operations. For example, while it is easy to establish whether compounds have been identified, it is harder to measure the quality or value of those compounds; and extremely difficult to measure the degree to which the organisational environment enables their discovery and maturation. Our approach is to identify and track the underlying critical success factors (CSFs) that deliver quality and optimisation of operations. Furthermore, we acknowledge the need to track the human, information and organisational capital that is required to sustain successful operations. When asked what they believed were the most important critical success factors in a Discovery organisation, Discovery leaders identified factors relating to both of these dimensions:
• Effective leadership
• People with the required capabilities
• Good working environment with appropriate human resource management
• Alignment between the external environment (commercial opportunities, competitors, regulatory bodies etc.) and internal operations (areas of focus, strategic objectives, goals etc.)
• An effective matrix for project teams
• Effective processes
• A good interface with Development.
More often than not, quantitative metrics will be hard to define for CSFs and some will need to be based on a quantitative scale with a designated baseline. While there is a necessary upfront investment of time and effort in order to define qualitative metrics (e.g. metrics around leadership or the working environment), this approach does enable an organisation to focus on factors which are likely to be better surrogate indicators of future success. This in turn provides further perspective on potential conflict between delivering near-term pipeline results versus developing the capabilities required to drive a successful discovery engine.
Furthermore, there are often tensions within the broad areas defined as CSFs and it is rarely easy to improve one aspect without compromising another, e.g. innovation vs strategic alignment, or effective processes vs good working environment. These must be recognised and aligned wherever possible in creating the necessary tracking tools. Similar tensions need to be recognised and managed in the overall approach to using metrics in the management of the organisation.
Summary
Metrics clearly have an important role to play in managing Discovery organisations for success. In developing metrics with our clients, our approach adheres to a few simple guidelines – keep the number of metrics in use at a manageable level, ensure that they are explicitly linked to and supportive of the goals of the organisation, and take into account co-dependencies between metrics when setting targets.
Finally, ensure that Discovery metrics are properly aligned with those used in functions external to Discovery, such as pre-clinical and early development.
Matthew McLoughlin is a vice president at Kinapse: matthew.mcloughlin@kinapse.com. Ann Baker is founding partner of Atkinson Cowan: ann@atkinsoncowan.com
For more information on Service Insight features contact InPharm’s sales team on +44 (0)1243 772 010 or email pharmafilesales@wiley.com
Related Content
PlasmidFactory founder Dr. Martin Schleef honoured with the NRW Innovation Award 2025
The founder and long-standing CEO of PlasmidFactory, Dr. Martin Schleef, was honoured in Düsseldorf with …
AAX Biotech announces collaboration for cardiovascular antibody therapy
Swedish biotech firm AAX Biotech has entered a new collaboration focused on the development of …
Research finds tablet effective in slowing progression of Alzheimer’s disease over 18 months
TauRx Pharmaceutics reports that hydromethylthionine mesylate (HMTM) could be an oral treatment for slowing the …
