A Unified Framework for Training-Dataset Aggregation and Exact Machine Unlearning via Gibbs Probability Measures
Abstract
In this paper, three operations on Gibbs probability measures are studied: renormalization, normalized log-linear combination, and nesting. The resulting measures are shown to be Gibbs probability measures with explicit expressions for their objective functions, reference measures, and regularization factors. These measures are further characterized as unique solutions to optimization problems involving linear combinations of the original objective functions, subject to either relative entropy regularization or relative entropy constraints. When the reference measures are probability measures, geometric interpretations of these three operations are established involving Pythagorean identities and exponential geodesics. A central result is the equivalence between nesting and normalized log-linear combinations of Gibbs probability measures. These results are then applied to supervised machine learning with Gibbs algorithms, which are represented by Gibbs probability measures. In this context, either through nesting or log-linear combinations, Gibbs algorithms trained on local datasets are used to form new Gibbs algorithms trained upon the aggregation of all local training datasets. Interestingly, the parameters of such operations, determine the weights of each training data point on the aggregated dataset. This observation is leveraged for achieving: (a) Exact machine unlearning in centralized, decentralized, and federated learning systems; and (b) Centralized-equivalent performance, in the sense of producing algorithms trained on the aggregation of all local datasets in decentralized and federated learning .
Related
- Training data attribution in diffusion models via mirrored unlearning and noise-consistent skew
- Generalized Bratteli diagrams without probability tail invariant measures
- Arithmetic probability measures
- An Improved Fractal Analysis of Directional Densities for Borel Probability Measures
- Invariant probability measures for path-dependent regime-switching processes
BibTeX
@misc{bermudez2026unified,
title = {{A Unified Framework for Training-Dataset Aggregation and Exact Machine Unlearning via Gibbs Probability Measures}},
author = {Yaiza Bermudez and Samir M. Perlaza and Iñaki Esnaola},
howpublished = {HAL},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.05544},
eprint = {2611.05544},
archivePrefix = {arXiv},
url = {https://inria.hal.science/hal-05851207v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - A Unified Framework for Training-Dataset Aggregation and Exact Machine Unlearning via Gibbs Probability Measures
AU - Yaiza Bermudez
AU - Samir M. Perlaza
AU - Iñaki Esnaola
PY - 2026
DA - 2026/11/03
PB - HAL
DO - 10.48550/arxiv.2611.05544
UR - https://inria.hal.science/hal-05851207v1
AB - In this paper, three operations on Gibbs probability measures are studied: renormalization, normalized log-linear combination, and nesting. The resulting measures are shown to be Gibbs probability measures with explicit expressions for their objective functions, reference measures, and regularization factors. These measures are further characterized as unique solutions to optimization problems involving linear combinations of the original objective functions, subject to either relative entropy regularization or relative entropy constraints. When the reference measures are probability measures, geometric interpretations of these three operations are established involving Pythagorean identities and exponential geodesics. A central result is the equivalence between nesting and normalized log-linear combinations of Gibbs probability measures. These results are then applied to supervised machine learning with Gibbs algorithms, which are represented by Gibbs probability measures. In this context, either through nesting or log-linear combinations, Gibbs algorithms trained on local datasets are used to form new Gibbs algorithms trained upon the aggregation of all local training datasets. Interestingly, the parameters of such operations, determine the weights of each training data point on the aggregated dataset. This observation is leveraged for achieving: (a) Exact machine unlearning in centralized, decentralized, and federated learning systems; and (b) Centralized-equivalent performance, in the sense of producing algorithms trained on the aggregation of all local datasets in decentralized and federated learning .
ER -