Workshop Advances PCORI’s Process for Picking High-Priority Research Questions

When PCORI Executive Director, Joe Selby, MD, MPH, opened our recent workshop on our process for selecting high-priority research questions to study, he told attendees "it would be a great understatement to say that we are excited about this day." I couldn’t agree more.

All of us who have been working on PCORI’s evolving research prioritization and topic generation initiative were pleased by the depth of discussion and provocative comments we heard from the 70 workshop participants -- expert methodologists, researchers, patients and other stakeholders. More than 100 others registered to join us via live webcast.

The goal of our research prioritization efforts is to identify the questions that are important to patients, their caregivers and clinicians so they can be the subject of PCORI funding announcements. Our December 5 workshop, held just outside Washington, D.C., was a critical part of that ongoing effort, which will employ several mechanisms to solicit and gather potential questions for study and then subjects the questions to a detailed, multi-stakeholder prioritization process.

One particularly noteworthy tool we’re using is our web-based “Suggest a Question” feature, which we launched in September and through which we’ve received more than 600 submissions so far. We’ve gathered an additional 800 questions through targeted engagement events and workshops.

We’re now refining a means of rigorously and systematically filtering these questions so we can start to choose those to fold into our funding announcement process. All of the questions we’ll eventually consider will fall under our National Priorities and Research Agenda. But we know we won’t be able to fund research on every question that makes that cut, so we need to make well-informed decisions on how best to spend our money.

Our research prioritization process is that filtering system. It will be implemented and used by our Advisory Panels. These multi-stakeholder panels will provide us with recommendations on which questions are most important for us to study in pursuing our research mission.

The process will involve reviewing each question using specific criteria developed for this purpose. The criteria include elements such as the likelihood that the research could change practice, the patient centeredness of the question, the impact or burden of the condition or the disease on the patient and on society, and the durability of the information – that is, the length of time the information would be expected to be useful.

We’re formulating our approach based on best practices from such research organizations as NIH, AHRQ and the IOM, a review of the scientific literature, guidance from PCORI’s Methodology Committee – and a test drive by a highly diverse multi-stakeholder pilot group. These 35 stakeholders walked through the process in October and November, using our criteria to prioritize 10 topics for research that we had selected. Topics included a range of conditions and diseases from back pain in the elderly to biomarkers for breast cancer and interventions to prevent obesity. The pilot group members joined us at the December 5 workshop to share their feedback and experience with the process.

Overall, there seemed to be broad appreciation of the complexity of research prioritization, but there was strong support for the need for such a process and for the approach we’re taking. We heard back from pilot participants on ways to simplify the process and shared thoughts on how to conduct prioritization with multi-stakeholder groups. We also heard from leading experts on prioritization methods, Dr. David Meltzer of the University of Chicago (a member of our Methodology Committee) and Dr. Claire McKenna of the University of York updated the audience on some of the latest developments in the area of value of information, a scientific approach to setting priorities for research.

We also heard from Dr. Gail Wilensky, a senior fellow at Project HOPE, who encouraged us to consider studying interventions associated with significant variations in practice across the country. “Those variations," she said, "have consequences,” whether in terms of patients' outcomes and well-being or in their economic implications.

We’ll be revising our draft prioritization process based on the feedback from the pilot participants and other participants of the workshop. We expect to have a draft report on our proposed process in hand by the end of January; we’ll make that available publicly along with a summary report on the workshop proceedings. As we work on that, we welcome your continued submissions to our “Suggest a Question” feature and your general comments.