Website CV Paper
Fields
Econometrics
Job market paper
Designing Experiments for Policies Beyond the Experiment
We study how to use pilot data to design experiments when the policy of interest cannot be directly implemented in the experimental setting. For instance, there may be noncompliance in the experiment, while the policy may treat the population as a whole. Accordingly, the payoff of the policy is only set-identified by the experimental data, so it is no longer clear what makes a good design. We answer this question from the perspective of ultimately implementing the policy. This leads to a two-wave statistical decision problem. We characterise the asymptotically optimal rules for choosing the experimental design and the subsequent policy. Evaluating a design requires integrating over the possible second-wave data and over the posterior of all the take-up rates and mean outcomes in the experimental arms. To facilitate implementation of optimal rules, we introduce regret-directed dimension reduction (RDDR), which we show reduces the dimension of integration while preserving the first-order asymptotic performance. In a numerical exercise calibrated to a Supplemental Nutrition Assistance Program (SNAP) outreach experiment, choosing the design with our rule reduces regret by 27% relative to the allocation in the original study's main analysis sample. RDDR retains nearly all of this improvement at about one-third of the computation time.