When Can We Work in Embedding Space? What Text Embeddings Preserve
Abstract
When do text embeddings work as inputs to empirical analysis? Their use rests on an assumption: that we can trade text for its low-dimensional embedding, and lose little in doing so. I make that assumption precise under a generative model in which documents are mixtures of latent topics. I study two uses---clustering units in embedding space and controlling for high-dimensional text. A cluster of embeddings is a set of documents with similar topic mixtures; controlling for the embedding is equivalent to controlling for the topic mixture, so validity reduces to whether that mixture captures the confounding. In an application to 363 U.S. metropolitan areas, embedding-based clusters of LLM-generated economic descriptions recover interpretable economic archetypes and separate local employment dynamics more sharply than clustering on model residuals, or on a curated set of industry and demographic covariates.
Related
- Improved computation of polynomial roots over number fields when using complex embeddings
- Uniform Embeddings of Banach Space Spheres into Hilbert Spaces and Property (H)
- Isometric Embeddings of Asymptotically Hyperbolic Manifolds into Hyperbolic Space
- An isometric embedding of the space of non-increasing sequences with non-negative terms into the Gromov-Hausdorff space
- Local Nevanlinna embedding for Hardy space of Dirichlet series below critical exponent
BibTeX
@misc{freyaldenhoven2026when,
title = {{When Can We Work in Embedding Space? What Text Embeddings Preserve}},
author = {Simon Freyaldenhoven},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.31059},
eprint = {2608.31059},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.31059},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - When Can We Work in Embedding Space? What Text Embeddings Preserve
AU - Simon Freyaldenhoven
PY - 2026
DA - 2026/08/31
PB - arXiv
DO - 10.48550/arxiv.2608.31059
UR - https://arxiv.org/abs/2608.31059
AB - When do text embeddings work as inputs to empirical analysis? Their use rests on an assumption: that we can trade text for its low-dimensional embedding, and lose little in doing so. I make that assumption precise under a generative model in which documents are mixtures of latent topics. I study two uses---clustering units in embedding space and controlling for high-dimensional text. A cluster of embeddings is a set of documents with similar topic mixtures; controlling for the embedding is equivalent to controlling for the topic mixture, so validity reduces to whether that mixture captures the confounding. In an application to 363 U.S. metropolitan areas, embedding-based clusters of LLM-generated economic descriptions recover interpretable economic archetypes and separate local employment dynamics more sharply than clustering on model residuals, or on a curated set of industry and demographic covariates.
ER -