Making Research More Patient Centered, One Application at a Time
PCORI’s mission is to support clinical comparative effectiveness research (CER) that helps patients and those who care for them make better-informed healthcare decisions. To accomplish this, we include patients and other stakeholders in all we do, including the merit review process we use to evaluate applications for PCORI funding. After all, what better way is there to ensure the projects we fund are truly patient centered than to include patients in the evaluation process?
While other funders have included patients and other stakeholders in their review processes, PCORI fully integrates them. We require patients and stakeholders on all our review panels and ask them to assess a wide range of criteria. And because we’re pioneers in advancing and funding patient-centered CER, to refine our process—and to provide evidence to other funders who wish to do the same—we are eager to share new evidence we have showing the difference these reviewers make.
We’ve published two papers (see box) that demonstrate how patients and other stakeholders contribute to merit review. We used analyses of review scores, reviewer surveys, and focus groups to study our process and how it affected both review outcomes and reviewer experiences. Unsurprisingly, we found that technical merit—whether a project is scientifically sound—is important to all types of reviewers and is the main factor driving which projects we fund.
Engaging Patients and Other Stakeholders in the Review Process
PCORI has published two papers that shed light on our inclusive merit review process that could prove useful to others in the field.
Researchers, Patients, and Stakeholders Evaluating Comparative-Effectiveness Research: A Mixed-Methods Study of the PCORI Reviewer Experience
Science Direct, October 2018
Unique Review Criteria and Patient and Stakeholder Reviewers: Analysis of PCORI’s Approach to Research Funding
Science Direct, October 2018
But we also found that all reviewers—patients, other stakeholders, and scientists—play an important role in informing which projects we fund, most notably through their ratings of the studies’ potential to improve real-world healthcare outcomes. As a stakeholder reviewer wrote, the diversity of reviewer types helped “illustrate whether the application was truly able to be successfully completed in the real world.”
Reviewers said our process resulted in more focus during review of other often-overlooked factors, including how useful a study’s results could be to patients and the plans for engaging with patients as partners during the research process.
As one patient reviewer wrote, “Patients and stakeholders were able to express practical aspects to the proposed studies—for example, to say whether expectations of study participants or clinicians were realistic.”
“Having a technically strong proposal is necessary for being considered for funding, but not sufficient,” one scientific reviewer wrote. “Strongly scored proposals also need to include patients, caregivers, and stakeholders’ perspectives and needs from the inception of the project all the way through the dissemination process.”
Patients and stakeholders were able to express practical aspects to the proposed studies—for example, to say whether expectations of study participants or clinicians were realistic.
Patient reviewer
Reviewers told us they felt like respected, equal partners in the review process. A few reviewers identified points they want us to keep working on, such as making sure that no one view dominates the conversation at in-person merit review meetings.
As a learning organization, we’ve continued to make enhancements to our process, based on the feedback from our reviewers and items we’ve learned on our own along the way. Some of the changes include expanding and standardizing our reviewer training and adding a criterion about the qualifications of the study team and research facility support. We will continue to study the implications of these changes on our work.
We’ve been encouraged to see early examples of other institutions—adding patient reviewers or new review criteria based on our approach. We understand that our commitment to engaging patients throughout the research process is a new one, and we will continue to lead efforts to encourage other funders to take it up in practice.