Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows

Abstract

In many high-stakes domains, extreme events carry substantial consequences, yet learning the heavy-tailed distributions that govern them from finite samples remains challenging: the quantities of interest are driven by a few extreme observations, so the tail is under-sampled relative to its importance. Even generative models tailored for heavy tails capture the tail region inadequately in practice. We propose the Conditional Value-at-Risk (CVaR)-penalized Generative Particle Algorithm (CVaR-GPA), a tail-agnostic algorithm for fine-tuning generative models toward heavy-tailed targets, built as a time discretization of the Wasserstein gradient flow of the Lipschitz-regularized KL divergence penalized by a CVaR discrepancy term. Such a flow can be initialized from the output samples of the pre-trained model, which are then transported along the gradient descent of the loss functional, without requiring access to the pre-trained model's internal architecture. The Lipschitz-regularized KL divergence requires minimal assumptions on the target, while the CVaR penalty focuses the flow on the tail. The CVaR penalty depends on the target only through a scalar tail statistic, inducing a velocity field that remains active in the under-sampled tail region at a dimension-free estimation cost. The resulting velocity field, and hence the depth of the transport map, implicitly adapts to the target, without target-specific modifications. Across four targets, including two real-world, high-dimensional datasets (daily streamflow in the Ohio River basin (d=64) and the Fama-French portfolios (d=25) with tail indices ranging from $1.05$ to $3.34$), fine-tuning with CVaR-GPA reduces global and tail errors by geometric-mean factors of 14.0 × and 9.8 ×, respectively, across seven pre-trained models spanning GANs, diffusion models, and other generative flows, with a single set of hyperparameters.

Wasserstein mirror gradient flow as the limit of the Sinkhorn algorithm
Nabarun Deb, Young‐Heon Kim, Soumik Pal +1
The Annals of Probability28 Aug 2026Peer-reviewedLicence not statedFull text19 citedGeneral mathematicsOpenAlex
On Over-Conservatism Under Low-Probability Extreme Events: A KL-Regularized Wasserstein Distributionally Robust Planning Method
Xinda Shen, Yifan Su, Bo Hu +3
arXiv13 Oct 2026PreprintarXiv licenceFull textOptimisation & controlarXiv
Control Strategies for Multi-Species Wasserstein Gradient Flows
Dante Kalise, Lucas M. Moschen, Grigorios A. Pavliotis
arXiv10 Sep 2026PreprintCC BY 4.0Full textAnalysis & PDEs Numerical analysis & scientific computingarXiv
Wasserstein accelerated proximal gradient algorithms
Kansei Ushiyama, Yi Feng, Kaito Fujii +2
arXiv2 Nov 2026PreprintCC BY 4.0Full textOptimisation & controlarXiv
Minimax Rates for Conditional Generative Learning under Wasserstein Loss
Sungwon Park, Kyoung-Sook Moon, Hongjoong Kim
arXiv26 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{gamage2026finetuning,
  title = {{Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows}},
  author = {Thejani Gamage and Hyemin Gu and Zhizhen Zhang and Ziyu Chen and Markos A. Katsoulakis and Luc Rey-Bellet},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2608.11544},
  eprint = {2608.11544},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.11544},
}

RIS

TY  - UNPB
TI  - Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows
AU  - Thejani Gamage
AU  - Hyemin Gu
AU  - Zhizhen Zhang
AU  - Ziyu Chen
AU  - Markos A. Katsoulakis
AU  - Luc Rey-Bellet
PY  - 2026
DA  - 2026/10/01
PB  - arXiv
DO  - 10.48550/arxiv.2608.11544
UR  - https://arxiv.org/abs/2608.11544
AB  - In many high-stakes domains, extreme events carry substantial consequences, yet learning the heavy-tailed distributions that govern them from finite samples remains challenging: the quantities of interest are driven by a few extreme observations, so the tail is under-sampled relative to its importance. Even generative models tailored for heavy tails capture the tail region inadequately in practice. We propose the Conditional Value-at-Risk (CVaR)-penalized Generative Particle Algorithm (CVaR-GPA), a tail-agnostic algorithm for fine-tuning generative models toward heavy-tailed targets, built as a time discretization of the Wasserstein gradient flow of the Lipschitz-regularized KL divergence penalized by a CVaR discrepancy term. Such a flow can be initialized from the output samples of the pre-trained model, which are then transported along the gradient descent of the loss functional, without requiring access to the pre-trained model's internal architecture. The Lipschitz-regularized KL divergence requires minimal assumptions on the target, while the CVaR penalty focuses the flow on the tail. The CVaR penalty depends on the target only through a scalar tail statistic, inducing a velocity field that remains active in the under-sampled tail region at a dimension-free estimation cost. The resulting velocity field, and hence the depth of the transport map, implicitly adapts to the target, without target-specific modifications. Across four targets, including two real-world, high-dimensional datasets (daily streamflow in the Ohio River basin (d=64) and the Fama-French portfolios (d=25) with tail indices ranging from $1.05$ to $3.34$), fine-tuning with CVaR-GPA reduces global and tail errors by geometric-mean factors of 14.0 × and 9.8 ×, respectively, across seven pre-trained models spanning GANs, diffusion models, and other generative flows, with a single set of hyperparameters.
ER  -

Abstract

Related

Cite