Yuya Shimizu

University of Wisconsin-Madison · Economics · 2026–27 job market

Fields

Econometrics, Applied Micro, Industrial Organization

Job market paper

Econometrics with Pre-Trained Embeddings for Unstructured Data

Unstructured data, such as images and text, are increasingly used in empirical economics. Since training machine learning models on unstructured data is costly, economists often use off-the-shelf pre-trained deep learning models developed by computer scientists to extract potentially high-dimensional embeddings, which are then used as covariates in target economic analyses. Despite the popularity of this practice, its theoretical foundations remain limited. There are two main difficulties. First, pre-trained models are typically trained on different datasets and for different tasks, making it unclear when they can be used validly for the target task. Second, the embedding function is not uniquely identified, complicating the analysis of its estimation error and the effect of that error on the target task. We provide sufficient conditions to overcome these difficulties. A key condition, which we call transferability, governs the convergence rate we derive. This rate accounts for source pre-training estimation error adjusted for the strength of transferability, as well as target estimation error with the pre-trained embeddings treated as standard covariates. To assess transferability, we develop a computationally feasible bootstrap test that does not require re-estimating the embeddings or nuisance functions. The transferability condition also provides practical guidance for applied researchers on selecting the source embedding function and target model classes. Our theory applies to a broad class of econometric analyses using double machine learning, including partially linear regression with unstructured controls, price elasticity estimation with product quality measured from images and text, missing-data imputation under selection on unstructured observables, and average treatment effect estimation with unstructured confounders. In an empirical application, we estimate the labor supply elasticity on Amazon Mechanical Turk, an online labor market platform, using embeddings of job descriptions as controls.