Judging success in funding medical research

Robert Pozen | September 6, 2011

Robert Pozen

Many charities are funding applied medical research -- trying to turn scientific discoveries into new treatments.

How can the governing boards of these charities determine whether their efforts have been successful?

It is not realistic to judge applied research on whether it produced a cure for a major disease.

But it also is not sufficient to say: "Our scientists and smart and hardworking, so continue to give them grants."

These governing boards need a performance-assessment system with specific objectives and measures with which to evaluate applied medical research.

Funding: Strong initial funding improves the chances of finding a break-through treatment.

Follow-on funding is usually a sign that an organization is achieving key milestones in a timely manner.

One important indicator is the degree of diversification of funding sources -- which lowers the risk of project cancellation or delay.

Another significant indicator is the portion of revenue consumed by administration, rather than research.

Talent: By recruiting the most talented researchers, an organization is more likely to generate new patient therapies.

As an organization shows fruitful results, it is better able to attract talented researchers.

Performance indicators should include not only the number and qualifications of its researchers but also the recruitment yield in response to its offers.

In addition, an organization should be prepared to terminate researchers who do not produce results as measured by other dimensions.

Creation: A key is to measure the volume of an organization's pipeline and its speed of progress toward milestones.

In addition, qualitative indicators should attempt to capture the organization's contribution to innovation through questions such as, "Has its work led to clinical trials that would not be otherwise conducted?"

The creation dimension also should encompass tools for applied medical research, such as animal models and databases.

Although tools do not directly improve patient care, they help other research organizations develop therapies.

Validation: This indicates the impact of an organization's output; it is often measured by articles published in peer-reviewed journals and citations to such articles.

Patent issuance is another form of validation.

Patent protection enhances the value of an organization's research by increasing the chances of commercial uptake.

Dissemination: Broadly, dissemination reflects the promulgation of findings.

This includes the dissemination of findings to the general public through websites or newsletters.

Dissemination goes further than scholarly articles; it encompasses releasing data to other researchers, and licensing patents to them on reasonable terms.

External uptake: This reflects the extent to which third parties choose to invest in the products of the organization's applied research.

These third parties may be for-profit companies, venture philanthropists or public-private partnerships.

These third parties are critical to the success of applied medical research; they have the expertise and resources to advance through human trials a concept proved in labs or mice.

Collaboration: Many characteristics of academia, such as promotion criteria and separate facilities, inhibit collaboration between researchers and clinicians. These barriers can be lowered by a successful applied research program.

Performance indicators for collaboration include co-authorships, data sharing and use of shared equipment. In addition, organizations should measure the ancillary impact of their work on other research institutions.

In short, as charities increase their funding of applied medical research, they should establish an effective system for performance assessment.

By articulating objectives and setting metrics, governing boards can build a consensus on their priorities and monitor progress in achieving these priorities.

Robert Pozen is senior lecturer at Harvard Business School and a senior fellow at the Brookings Institution; Heather Kline, who in 2011 received an MBA from Harvard Business School, assisted in this column, which summarizes a longer article they wrote that was published in Science: Translational Medicine.

Comment on this article