Measuring the Reach of PCORI's Funded Research Results

As patients, clinicians, researchers, and other members of the healthcare community know all too well, research findings can sit in the scientific literature, taking far too long to help improve patients’ lives—if they ever do.

Since its inception, PCORI has been working to assure that findings from our funded research will do better at reaching patients, clinicians, and other decision makers. The first step in our research process is to identify questions that matter most to healthcare stakeholders, particularly to patients. We then fund projects that—with stakeholder guidance—seek to answer those questions. Finally, we work diligently to get project results to those who can use them to make more-informed healthcare decisions.

We think studies and results most important to healthcare stakeholders will be taken up and used more rapidly than has traditionally been the case. And we have early data that show our funded projects’ results are getting out there—quickly, and to a lot of people.

Making Research Results Available

Once projects are finished, our funded researchers submit detailed reports on how they conducted their studies, and PCORI produces lay-language and professional summaries of their projects’ findings. All of those products are posted on our website and are accessed by many readers. In the final quarter of 2019, the lay and professional summaries were viewed nearly 29,000 times on pcori.org, up from just under 17,000 during the first quarter of 2019. Since PCORI posts the findings of all completed studies, this number will continue to grow over time.

In addition to these reports and summaries, PCORI-funded researchers are also publishing in peer-reviewed journals. As of December 2019, nearly 250 PCORI-funded projects had published comparative clinical effectiveness research (CER) results, up from 200 just three months earlier. As PCORI’s relatively young research portfolio continues to mature, the number of publications will increase rapidly.

An analysis of PCORI’s early funded studies found that almost 60 percent published results within 30 months of the project’s completion date. In contrast, an analysis in The BMJ found that typically only about one-third of research studies publish their results over that same time period, and only about two-thirds of them ever publish their results. That lag in sharing results is what PCORI wants to change.

Maximizing Access to Results

Beyond getting results posted on our website and published in peer-reviewed journals, it’s important that people have access to the articles in scientific journals. Many journals require a subscription to access all their articles. Open access—the practice of making peer-reviewed scholarly research and literature freely available online to anyone interested in reading it, without a paywall or subscription—is one way to ensure wide access to results papers.

Papers published as open access, or made available through public-access provisions, can get more and different readership. With this in mind, PCORI will, at an awardee’s request, pay the fees to provide free public access to peer-reviewed journal articles that present findings from its funded research.

Additionally, we use metrics to track the value of each publication about our funded work. For example, papers about PCORI-funded projects, on average, are published in journals that perform better than 90 percent of all journals on a traditional measure called the journal impact factor. This means that PCORI articles appear in more prestigious journals that tend to be frequently cited in other research.

Since traditional metrics often only speak to how publications are disseminated within the research community, we also look at alternative metrics—called altmetrics—that analyze the attention that papers receive in practical, everyday platforms. This includes article views and downloads, mentions in news outlets, and social media discussion. We’re thrilled to see that 17 percent of PCORI’s CER results articles have very high altmetrics scores, compared to only 1 percent of all medical and health articles.

Papers about PCORI-funded projects, on average, are published in journals that perform better than 90 percent of all journals on a traditional measure called the journal impact factor. This means that PCORI articles appear in more prestigious journals that tend to be frequently cited in other research.

Just the Beginning

In addition to publishing their findings in prominent journals, our awardees have given more than 5,000 presentations, including invited talks, peer-reviewed presentations, webinars, and more. Nearly two-thirds of our funded projects’ researchers have given at least one presentation so far, with more than 40 percent including a patient or another stakeholder partner as a presenter. This serves to both humanize the research and explain how it is patient centered. Our researchers have presented to diverse audiences, which means they are reaching far beyond the traditional research community (see graphic).

And, as you’re likely aware, PCORI provides multiple Dissemination and Implementation funding opportunities to support projects that actively facilitate the uptake and integration of evidence from PCORI-funded studies, in the context of related evidence, into real-world practice.

We have early indications that the results from PCORI-funded studies are on track to be useful and make an impact. We’ll update you on our progress as our funded research portfolio continues to mature.

Data from major medical journals show that @PCORI-funded research results are getting to the public faster than results from traditional research. https://bit.ly/2UP7jy2