Pharma Success Rates 1997-2006

Interesting paper looking at the success rates of 77 pharmaceutical companies from 1997-2006

The innovation hit rate, or success rate, has important implications. If concerns over losses lead to attempts to avoid failure, the firm will also avoid learning (Corbett, Neck & DeTienne, 2007; McGrath, 1999). If firms can learn from past failures and improve the chances of future endeavors, then the hit rate may contribute to perceptions of learning efficacy (Garrett, Covin & Slevin, 2009). Sitken (1996) suggests that repeated success can lead to delusions that everything is fine even if the process could benefit from adjustments. Anecdotal evidence suggests that a series of misses can dampen employees’ enthusiasm for persisting or managers’ interest in allocating resources for innovation, while a string of hits may build a firm’s confidence or attract investors (Kanter, 2006).

Key findings:

• as the number of projects in the portfolio increases, the launch rate decreases
• an inverted-U curvilinear relationship between the percentage of projects that are newly initiated and the launch rate
• the launch rate decreases with increases in the percent of new projects in knowledge categories that are new to the firm.

Green, KR 2013 Experience and the Hit Rate for Entrepreneurial Initiatives American Journal of Business and Management Vol. 2, No. 2, 2013, 91-105

Fast-Fail Word Cloud

1- argues associated assume atom attrition comments compare continue costs cycle decision delivered denoted development discovery drug ec email entire er et example expected facebook fail false-negative false-positive figure follow full gap generally given i2 increasing incurring industry innovation later marketable model molecules months opportunity outperforms parallel pass performed pharmaceutical placing possible posts potentially preview probability process productivity progress project quick-kill quickly range rates rdp reduce relative risk serial share shift simply sneak speed stage stochastic strategy successful taken tasks terminate test theory times twitter unmarketable used value worked year

created at TagCrowd.com

Opportunity Costs in Drug Development

In drug development, the marginal development costs associated with a decision to continue development may seem trivial compared to the potential losses associated with a decision to terminate a potentially marketable molecule.

For this reason project teams tend to continue with development even in the face of mounting evidence that drug development should be terminated.

While a decision to continue development may be easier for the project team, there are significant opportunity costs associated with the continued development of a molecule which is then terminated later in the development process.

And in the absence of infinite resources, by choosing to develop a molecule which is unmarketable we deny marketable molecules the chance to be evaluated and deny patients access to novel treatments.

Thought For The Day

What if portfolio management and the entire world based around expected net present value had zero predictive validity?

See Goodhart's Law

Drug Discovery as a Stochastic Process

The Quick-Kill model assumes drug discovery is a stochastic process. 

Stochastic models assume that for any given molecule there is a probability, p, that it is potentially marketable – safe, effective, meets an unmet clinical need, generating a commercial return on investment. 

The objective of pharmaceutical development is to advance such molecules as quickly as possible. The remaining proportion, 1-p, molecules are unsuitable for pharmaceutical development.

The Quick-Kill model seeks to eliminate, or “kill”, these remaining molecules as quickly as possible.

The model assumes there are m independent stages denoted by i.

At each stage i, the financial and time costs can be denoted as C_i and T_i. 

At the end of each stage, i, in the discovery and development process we make a go/no-go decision to progress a compound.

Thus, at each stage i, there are four possible outcomes: 

Progress a potentially marketable molecule to the next stage – quick wins.

Terminate a potentially marketable molecule so it does not progress to the next stage – a false negative.

Progress an unmarketable molecule to the next stage where it will fail at that, or a later, stage – a false positive.

Successfully terminate an unmarketable molecule – a quick-kill.

In an ideal world, we would progress only marketable molecules – winners - and terminate unmarketable molecules as quickly as possible in the development process - quick-kills.
 

R&D Parallelism: Worked Example

Sneak Preview

In the following worked example C1=C2 and T1=T2.

The cost of each test is, say, US$ 300k and the time taken to complete each test is, say, 6 months.

Thus the cost of a full cycle of the Serial Process is US$ 600k and the time taken to complete a full cycle is 12 months.

Compare this to a proposed development speed initiative in which the two tests are performed in parallel.

This Parallel Process reduces the time taken for successful molecules to pass through this stage of development since the decision to continue or terminate development is made after just 6 months – half that of the 12 months for a full cycle of the Serial Process.

Note that the full cycle cost is the same for both the Serial Process and the Parallel Process at $US 600k in total. However, in the Serial Process, if a decision is taken to terminate development at the end of Test 1 then the cost of performing Test 2 is avoided.

In this worked example, we use a range of probabilities from 5% to 50% to compare the relative performance of the two strategies. The RDP values (number of marketable molecules passed to the next stage in development per $USm years) were calculated for the Parallel Process and the Serial Process.

Figure 1 shows RP, the relative productivity of the Parallel Process to the Serial Process as a function of the probability of success, p, for the Parallel Process and for the Serial Process given a range of false-positive (α) and false-negative (β) rates. Values greater than 100% indicate the Parallel Process is more productive than the Serial Process. Values less than 100% indicate the Serial Process is more productive than the Parallel Process.

INSERT FIGURE 1

As the probability that the molecule is a marketable molecule increases then, as might be expected, the relative efficiency of the Parallel Process increases.

However, for a range of choices of α and β the Serial Process outperforms the Parallel Process. In particular, note that the Serial Process generally outperforms the Parallel Process across the entire range from 5% through to 15%.

R&D Parallelism: Reprise

Sneak Preview

These results demonstrate that placing development tasks in parallel to minimize the cycle time of successful molecules and increase development speed may simply increase R&D costs and actually reduce R&D productivity – the Development Speed Paradox.

In particular for high risk processes such as pharmaceutical R&D a Serial Process generally outperforms a Parallel Process across the entire range of success probabilities from 5% through to 15%.  This encompasses the widely accepted range of industry estimates for the probability of successful development, p, used by Paul et al. 2010 [4].

And if the definition of success is further restricted to those molecules which go on to be approved and generate a commercial return on investment then the Serial Process is likely to outperform a Parallel Process for much of the pharmaceutical research and development process.

This effect is large and robust to a range of probabilities, development costs, development times, false-positive and false-negative rates.

The exact point at which a Parallel Process will begin to outperform a Serial Process will depend upon the likely attrition rates at each stage in development and the sensitivity and specificity of the tests performed during each of those stages. 

These findings can be extended to multiple development tests and multiple development decision points[12].

This effect may be observed even if the Serial Process incurs additional financial and time penalties.

Such penalties might be incurred if the first test was performed with, say, a non-IND formulation making additional formulation work essential prior to a later test. 

The key take-home message is that increasing development speed by placing tasks in parallel may reduce R&D productivity.

Increasing development speed to increase R&D productivity can be likened to increasing production rates to reduce quality issues in manufacturing.  It is expensive and generates a lot of waste[14] . 

In R&D this waste is measured as late-stage attrition.

Minimizing cycle-time without addressing attrition rates meant the industry simply became really slick at delivering late-stage failures to the market place, precipitating the current innovation crisis in pharmaceuticals. 

The debate continues about the actual scale of this crisis[15].  However, there can be little doubt that by focusing on maximising the development speed of successful molecules, the industry may have optimized the drug development process around that tiny minority of molecules that make it to market.  Efforts to maximize development speed may simply have clogged the development pipeline with marginal or failing medicines incurring massive opportunity costs.  

Lendrem, DW 2013