Pierce Plucker

Oklahoma State University · Economics · 2026–27 job market

Fields

Econometrics, Urban & Regional, Development

Job market paper

A Model Confidence Set for Learner Selection in Generic Machine Learning

Generic Machine Learning (Chernozhukov et al., 2025a) provides valid inference on heterogeneous treatment effects using any machine learning algorithm to proxy the conditional average treatment effect, but offers limited guidance on which learner's results to report when multiple learners are employed. Existing reporting practice typically defaults to ad hoc selection of a top-performing learner or to a Bonferroni correction across all candidates, neither of which adapts to the relative performance of the learners in the data. I propose applying the model confidence set of Hansen et al. (2011) to the split-level goodness-of-fit values produced by GML, yielding a data-driven set of learners whose performances are statistically indistinguishable, a set that collapses to a singleton when one learner clearly dominates. Inference then proceeds over this set rather than the full collection of candidates. An initial split of the sample reserves a portion of the data exclusively for learner selection, so that the reported inference uses only observations that never entered the selection stage. The resulting two-stage procedure carries an overall coverage guarantee of 1 − α1 − α2 , with separate budgets for selection and inference error. Monte Carlo simulations show that the procedure controls size, sacrifices little power relative to naive selection, and reveals variation in point estimates across surviving learners when the data do not support a unique choice. In an application to the lending experiment of Bryan et al. (2024), formal learner selection recovers a significant boost in profits for the top quartile of firms at the conventional 90% level that is otherwise lost to multiplicity correction across all candidate learners. A second application to the groundwater conservation experiment of Hrozencik et al. (2024) illustrates a complementary case: the MCS identifies a relatively superior learner, while subsequent GML inference provides little evidence that even the selected proxy captures systematic treatment-effect heterogeneity. Together, the applications show that formal learner selection can sharpen inference when informative proxies exist without manufacturing evidence of heterogeneity when they do not.