Econometrics with Pre-Trained Embeddings for Unstructured Data
Abstract
Unstructured data, such as images and text, are increasingly used in empirical economics. Since training machine-learning models on unstructured data is costly, economists often use off-the-shelf pre-trained deep learning models developed by computer scientists to extract embeddings, which are then used as covariates in target economic analyses. Despite the popularity of this practice, its theoretical foundations remain limited. There are two main difficulties. First, pre-trained models are typically trained on different datasets and for different tasks, making it unclear when they can be used reliably for the target task. Second, the embedding function is subject to an identification problem, complicating the analysis of its estimation error and the effect of that error on the target task. We provide sufficient conditions to overcome these difficulties. A key condition, which we call transferability, governs the convergence rate we derive. To assess transferability, we develop a computationally feasible bootstrap test that does not require re-estimating the embeddings and nuisance functions. Our theory applies to a wide range of double machine learning applications, including partially linear regression with unstructured controls, price elasticity estimation in demand models accounting for product quality measured by images and text, missing-data imputation using unstructured data, and average treatment effect estimation with unstructured confounders. As an empirical application, we estimate the labor supply elasticity on Amazon Mechanical Turk, an online labor market platform, using job-description embeddings as controls.
Related
- Tuning Parameter Selection in Econometrics
- Can Geopolitical Risk Improve the Forecasting of Thai Stock-Market Returns? Evidence from Econometrics and Machine-Learning Models
- How to identify factors influencing mode choice behaviors within a heterogeneous metropolitan area? A novel approach midway between data science and econometrics
- Exponential stability of port-Hamiltonian systems with unstructured, measurable Hamiltonian densities
- Application of Machine Learning to the Computation of Parameters in the SUPG Method for Unstructured and Anisotropic Meshes
BibTeX
@misc{shimizu2026econometrics,
title = {{Econometrics with Pre-Trained Embeddings for Unstructured Data}},
author = {Yuya Shimizu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.17378},
eprint = {2607.17378},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.17378},
}
RIS
TY - UNPB
TI - Econometrics with Pre-Trained Embeddings for Unstructured Data
AU - Yuya Shimizu
PY - 2026
DA - 2026/09/25
PB - arXiv
DO - 10.48550/arxiv.2607.17378
UR - https://arxiv.org/abs/2607.17378
AB - Unstructured data, such as images and text, are increasingly used in empirical economics. Since training machine-learning models on unstructured data is costly, economists often use off-the-shelf pre-trained deep learning models developed by computer scientists to extract embeddings, which are then used as covariates in target economic analyses. Despite the popularity of this practice, its theoretical foundations remain limited. There are two main difficulties. First, pre-trained models are typically trained on different datasets and for different tasks, making it unclear when they can be used reliably for the target task. Second, the embedding function is subject to an identification problem, complicating the analysis of its estimation error and the effect of that error on the target task. We provide sufficient conditions to overcome these difficulties. A key condition, which we call transferability, governs the convergence rate we derive. To assess transferability, we develop a computationally feasible bootstrap test that does not require re-estimating the embeddings and nuisance functions. Our theory applies to a wide range of double machine learning applications, including partially linear regression with unstructured controls, price elasticity estimation in demand models accounting for product quality measured by images and text, missing-data imputation using unstructured data, and average treatment effect estimation with unstructured confounders. As an empirical application, we estimate the labor supply elasticity on Amazon Mechanical Turk, an online labor market platform, using job-description embeddings as controls.
ER -