Missing Data in PCOR Studies: Funding Methodological Research to Address a Pervasive Problem

Researchers conducting clinical trials or observational studies often find that they are missing data. A clinician forgets to order a test, a technician fails to record a result, a patient drops out of the study. When it comes time to analyze the data, these gaps complicate the task. To reach reliable conclusions, researchers need to know whether the data is missing by chance or whether there is an underlying cause.

How to deal with missing data is a priority area for PCORI’s CER Methods and Infrastructure program, which aims to fund high-impact studies that address gaps in methodological research and lead to improved methods. Missing data is a widespread problem in all observational studies and clinical trials. Without special analytical methods, unbiased results can’t be obtained from studies that are missing data on some patients. The essential problem is that researchers must make assumptions about the nature of the causes of the missing data to arrive at inferences about treatment effects. Our Methodology Standard MD-5 requires that researchers pay attention to their assumptions about the causes of missing data.  

The CER Methods and Infrastructure program has funded several projects addressing important problems related to missing data:

• Manisha Desai and colleagues at Stanford University are examining how to deal with missing data in longitudinal studies with time-varying covariates (TVCs). TVCs are variables that can change value over the course of the observation period. Such variables might include body weight, income, or marital status. The team plans to develop an algorithm with open-source software for simulating studies with such variables and create methods for handling that type of missing data.

• Qi Long and colleagues at Emory University are developing new multiple imputation methods for missing data in large observational studies (imputation is the process of replacing missing data with substituted values). They will apply resulting methods to stroke registry data used to identify patient and hospital characteristics that are associated with quality of care.

In analyzing clinical trials, one of the most important problems related to missing data is dropout—when study participants leave the study prematurely. Daniel Scharfstein and colleagues at Johns Hopkins University are developing and testing open-source software for conducting novel sensitivity analyses for repeatedly measured outcomes with dropout. If you would like to become a user of the software, participate in the online discussion forum, or collaborate on case studies with these investigators, I encourage you to visit the project’s web portal.

The CER Methods program has made dissemination of research results a key consideration in our funding decisions. All three of these projects include the development and dissemination of open-source software products that will facilitate use in patient-centered outcomes and comparative effectiveness research. We anticipate that the methods developed through these awards will improve researchers’ ability to analyze the data collected in randomized trials and observational studies, and thereby improve the evidence base for clinical and health policy decision making.