Dingli Wang

University of California, Riverside · Economics · 2026–27 job market

Fields

Econometrics, Finance, Innovation

Job market paper

Robust Replacement of Human Decision Makers under Distribution Shift

This paper studies when a machine learning algorithm should replace a human decision maker under distribution shift. We infer each human's error rates from past decisions, assuming that they stay the same on future cases, while the data seen by the algorithm may shift within a Wasserstein ball. For each threshold, we compute the algorithm's worst false positive and true positive rates over the ball, which gives a worst-case ROC curve. We replace a human only if some point on this curve is, with high posterior probability, at least as good as the human in both rates. We show that the estimated curve converges uniformly and bound the welfare loss from replacement. We also compare the rule with an infeasible benchmark that knows the deployment distribution and each human's error rates and sets the algorithm's false positive rate equal to the human's. Relative to this benchmark, choosing the threshold with the highest posterior probability can leave a welfare loss that does not vanish with more data. In simulations, robust rules replace fewer humans than a rule that ignores shifts, giving up some gains without shifts but losing less after large shifts.