Jake Carlson

Harvard University · Economics · 2026–27 job market

Fields

Econometrics

Job market paper

Interpretable Discovery from Unstructured Data: A High-Dimensional Approach

I propose an automatic, general-purpose framework for making discoveries from unstructured data (e.g., text data from open-ended surveys of economic beliefs). The framework leverages recent methods from the literature on AI interpretability to transform unstructured datasets into high-dimensional, structured datasets of interpretable concept measures; specifies parameters and null hypotheses of interest based on these concept measures; tests these hypotheses using algorithms validated by new results in high-dimensional multiple testing, producing a selected set (“discoveries”); and automatically generates and evaluates natural language descriptions of the concepts associated with these discoveries. The proposed framework has few researcher degrees of freedom, is robust to data snooping, mitigates under-exploration, and facilitates fast and inexpensive sensitivity analysis and replication. I revisit applications to recent descriptive and causal analyses of unstructured data in empirical economics, and find this framework is able to automatically replicate existing discoveries, add nuance to others, and make entirely new discoveries as well.