Quantifying Behavioral Tails in Black-Box Language Models
Abstract
We introduce RareTrap, a framework for estimating the probability of severe behaviors in black box large language models (LLMs). A key challenge for probability estimation is defining a tractable distribution over the input space. To accomplish that, RareTrap uses a surrogate LLM and constructs a geometry-aware mapping from a lower-dimensional latent reference space into its token-embedding space to induce an explicit and reproducible distribution over input prompts. A response-level performance function is utilized on the response to quantify behavior severity. This enables sequential rare event simulation that concentrates evaluations on progressively more severe behaviors while preserving probability under the induced prompt distribution, which would otherwise be prohibitive to measure. Across 10 open-weight and two frontier models (GPT-5.4 and Claude Sonnet 4.6), we find that RareTrap successfully induces severe resource consumption behaviors and computes their probability with as few as 200 evaluations. RareTrap provides model developers a principled approach for evaluating language models under a common distribution, and prioritizing alignment effort to improve safety and mitigate risks.
Related
- Threshold Tails of Black hole Differential Observables
- A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models
- Large-Scale Comparison of Metaheuristic Algorithms via Behavioral Verification of Large Language Model-Generated Codes
- Diffusion Language Models for Perplexity-Aware Black-Box Attacks
- An invariant variational proof for canonical processes with independent symmetric log-concave tails
BibTeX
@misc{eshra2026quantifying,
title = {{Quantifying Behavioral Tails in Black-Box Language Models}},
author = {Elsayed Eshra and Ali Al-Lawati and Dongwon Lee and Suhang Wang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.33638},
eprint = {2609.33638},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.33638},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Quantifying Behavioral Tails in Black-Box Language Models
AU - Elsayed Eshra
AU - Ali Al-Lawati
AU - Dongwon Lee
AU - Suhang Wang
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.33638
UR - https://arxiv.org/abs/2609.33638
AB - We introduce RareTrap, a framework for estimating the probability of severe behaviors in black box large language models (LLMs). A key challenge for probability estimation is defining a tractable distribution over the input space. To accomplish that, RareTrap uses a surrogate LLM and constructs a geometry-aware mapping from a lower-dimensional latent reference space into its token-embedding space to induce an explicit and reproducible distribution over input prompts. A response-level performance function is utilized on the response to quantify behavior severity. This enables sequential rare event simulation that concentrates evaluations on progressively more severe behaviors while preserving probability under the induced prompt distribution, which would otherwise be prohibitive to measure. Across 10 open-weight and two frontier models (GPT-5.4 and Claude Sonnet 4.6), we find that RareTrap successfully induces severe resource consumption behaviors and computes their probability with as few as 200 evaluations. RareTrap provides model developers a principled approach for evaluating language models under a common distribution, and prioritizing alignment effort to improve safety and mitigate risks.
ER -