About 80 percent of studies of U.S. medical interventions use randomized controlled trials (RCTs), the gold standard of laboratory research. But only about 18 percent of studies of U.S. health care delivery use RCTs. That can and should change, suggests Amy Finkelstein, the Ford Professor of Economics at MIT, in a Science piece co-written with MIT researcher Sarah Taubman. If so, they assert, researchers who find new ways of applying RCTs to our medical system will be able to produce compelling answers to pressing questions. Finkelstein, an experienced practitioner of RCTs in U.S. health care, helped launch J-PAL North America, a recently formed branch of MIT’s Abdul Latif Jameel Poverty Action Lab, which uses RCTs to test policy questions and social programs. MIT News discussed the issue with Finkelstein.
Q. In the Science piece, you say that randomized evaluations of policy and delivery should be “closer to the norm than the exception.” What is the power of randomized evaluations as a research method?
A. It is not always obvious what the effect of a given policy is. For example, what is the impact of covering the uninsured with health insurance? Comparisons of the insured and the uninsured often indicate that those with insurance are in worse health than those without insurance. Would it be right to conclude, therefore, that health insurance makes individuals sicker? Or, more reasonably, are individuals who are in poor health more likely to seek out health insurance?
Random assignment solves this problem of inference. In randomized evaluations, individuals are randomly selected to receive an intervention, such as health insurance. Those individuals who are not selected form a comparison group. Because the selection process is random, the two groups are similar in every respect, except that one group receives the intervention, while the other does not.
Therefore, if after the intervention is implemented, the group that received the intervention has different outcomes — is more or less healthy, or has higher or lower medical expenditures — we know that these differences were caused by the intervention.
This clear attribution of what is due to the intervention is the key strength of randomized evaluations. The results are transparent, easy to explain, and credible. The resulting discussion can focus more comfortably on the implication of the results, rather than on the methods and their validity.
Q. What are the main barriers to developing more randomized trials in health care — and how can they be addressed?
A. One common concern is that randomized evaluations require substantial resources, both in terms of time and cost. It is important to distinguish between the costs of prospective research in general, and the [added] cost of doing that prospective research through a randomized design.
The standard model for randomized controlled trials in medical interventions is expensive and time-consuming. Historically, most randomized evaluations in health care delivery have followed this same model, which involves screening, recruiting, and obtaining informed consent from individual subjects before randomly assigning them to a treatment or control group. This process, in combination with collecting primary follow-up data, can be difficult and labor-intensive.
But we suggest there is an alternative and less expansive approach for many randomized evaluations in health care delivery, where safety is often not an issue —as it may be in medical trials. Randomization of who is offered an intervention can be conducted on a set of potentially eligible individuals with a waiver of informed consent. This approach allows for larger trials with more representative samples, because it does not require individual recruitment. At the same time, it does not interfere with estimating an intervention’s causal effect, even if there is imperfect adherence, as when not all those offered the intervention choose to enroll.
In addition, individuals can be followed passively in administrative data, which is used and stored for reasons other than the study — such as for insurance claims, hospital discharges, or electronic medical records. This can often allow researchers to examine a wide range of impacts at substantially lower cost than primary data collection.
Randomized evaluations are particularly appropriate when programs are oversubscribed, rolled out in a gradual fashion, or initially tested with pilot programs. In those cases, randomization can be seen as the fairest way of determining participation, while simultaneously allowing for rigorous measurement of the program’s impact. The Oregon Health Insurance Experiment — a randomized evaluation of the impact of covering low-income uninsured adults with Medicaid, which Harvard’s Katherine Baicker and I have been leading — came about because the state of Oregon decided that random selection by a lottery was the fairest way to allocate a limited number of Medicaid slots.
Q. You would like to see more randomized evaluations that are actively designed by researchers themselves. How can this help us develop clearer, more specific answers to major health care delivery issues?
A. There is a large range of open questions in health care delivery that have the potential to be evaluated through RCTs. One natural area is in insurance design, where there have already been a few large RCTs on the effect of individual coverage, such as the 2008 Oregon Health Insurance Experiment and the RAND Health Insurance Experiment from the 1970s. Some important current issues in insurance design that could be studied through randomized evaluation concern the impacts of how insurers reimburse providers — such as the possibility of paying more for higher-value care, or covering a limited network of providers.
Another natural area for randomized evaluations concerns the impact of interventions designed to bring health care practice more in line with consensus recommendations. In almost every area of medicine, there is evidence that individuals do not receive all the care that is recommended, while receiving care that is not recommended. One can imagine a wide range of potential randomized interventions across patients, and/or across providers, to examine the impact of interventions designed to bring practice more in line with recommendations. These could make use of tools including financial and nonfinancial incentives, information, defaults, and nudges, as well as decision-support tools. One could study not only the impact of these different tools on compliance with recommended care, but also, in turn, the impact of that recommended care on “downstream” health care use and health outcomes.
Of course, not everything is appropriate for study through randomized evaluation. For example, if we are interested in how marketwide changes in health insurance coverage — such as the major expansions of Medicaid under the Affordable Care Act — affect health care use and health outcomes, a randomized evaluation of covering individuals with insurance may not capture the full systemwide effects, and randomizing expansions across markets is unlikely to be feasible or appealing.
At the same time, however, some system-level interventions can be fruitfully studied through random assignment. For example, innovation in the payment structure for health care providers, including bundling payments for episodes of care and creating shared saving contracts, is emerging as a major theme in health policy. As these marketwide payment mechanisms expand to take on new groups of patients, one could randomize which patients are included in order to study some of their impacts.
